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ABSTRACT 

This  thesis  describes  an  experiment  conducted  at  the  Naval 
Postgraduate  School  (MPS)  during  the  perioa  15  October  1905 
through  20  October  1585.  Specifically,  the  experiment  evaluates 
"pull-down  window"  n  i c r o c o m p u t e r  technology,  continuous  speech 
recognition  equipment,  and  standard  computer  keyboard  entry  to 
input  commands  in  a  command  and  control  environment.  Using  the 
Naval  Warfare  Interactive  Simulation  System  (NWISS)  as  a  controlled 
medium,  military  problems  were  posed  to  test  subjects  in  specific 
light  and  noise  environments. 

Although  the  results  are  not  entirely  conclusive,  they  go 
demonstrate  a  distinct  advantage  in  using  continuous  speech  or 
keyboard  entry  modes  over  the  drop-down  window  technology  of  the 
Macintosh  (if  subject  training  time  is  not  a  significant  restric- 
tion). Either  the  continuous  speech,  or  the  keyboard  method  was 
clearly  superior  in  all  environments. 
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I.   INTRODUCTION 

A.   OVERVIEW 

Computer  Technology  has  increased  exponentially  ever  the  past 
several  years.  As  a  result  •  work  is  now  being  completed  more 
efficiently  and  inexpensively  than  even  before.  Nevertheless t 
faced  with  a  myriad  of  available  computer  optionsi  it  behooves 
the  individual  user  to  choose  one  that  is  not  only  user  frienaly, 
but  is  both  cost  effective  and  a  proven  performer.  Time  has 
always  been  a  critical  factor  for  any  business,  and  people  are 
now  realizing  that  using  a  computer  only  constitutes  the  first 
step  toward  achieving  reductions  in  this  area. 

In  recent  years  there  has  been  a  movement  afoot  to  simplify 
the  all  important  man-machine  interface  which  counts  so  heavily 
in  on-line  efficiency  computations.  While  the  computer  keyboard 
has  long  been  the  industry  standard,  its  speea  is  severely  limited 
by  the  skill/training  of  the  user;  and  the  subject's  knowledge  of 
the  particular  command  keys.  Due  primarily  to  these  recognizee 
limitations,  more  attention  is  being  paid  to  a  faster,  anc  more 
sophisticated  means  of  communication  between  user  ana  machine: 
Automatic  Speech  Recognition  (ASR).  This  concept  was  first  introduced 
in  the  late  1950' s.  and  significant  progress  was  made  wit bin  the 
last  15  years.  It  is  known  that  Tree  speech,  in  its  natural 
form,  is  the  fastest  and  most  economical  means  for  homo  sapiens 
to  communicate  amongst  themselves.  Applying  this  fact  to  the 
user-machine  interface  cilemma,  it  has  been  shown  that  rnacnines 
can  not  only  be  taught  to  respond  to  any  language,  but  that 
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speech  for  this  type  of  cor.ir.unicati.cn  leaves  the  user's  hands  and 
feet  free  in  order  to  perform  important  concurrent  functions 
(i.e.,  viewing  Graphics  screens  cr  decision  support  aics,  personnel 
interface,  etc.).  while  admittedly  the  progress  toward  an  econom- 
ically feasible  voice  recognition  system  has  been  slow,  ASR 
products  are  now  making  themselves  known  in  industrial,  military 
and  security-oriented  applications.  It  is  this  encouraging  trend, 
coupled  with  the  significant  accomplishments  in  each  area  of  use 
that  leads  researchers  to  believe  in  its  unlimited  potential. 
[Ref.  1  and  2] 

The  newest  development  in  interface  technology  is  the  "pull-down 
window"  incorporated  in  the  Macintosh  microcomputer.  (See  Enclosures 
1-6  in  Appendix  G.)  This  methodology,  in  conjunction  with  the  use 
of  a  mouse  peripheral  input  device,  presents  a  "rectangular-shaped 
menu"  on  a  computer  screen  within  which  the  user  writes  and  reads 
data  representing  a  program  in  the  computer.  Because  the  mover ent 
of  the  cursor  matches  the  movement  of  the  mouse,  the  user  r.eeo 
only  place  the  cursor  on  the  oesired  command  and  press  a  conveniently 
located  button  for  instant  activation.  Although  no  comparisons 
(individually  cr  collectively)  have  currently  been  made  with  the 
more  traditional  interface  devices  because  of  its  recent  emergence 
in  the  marketplace,  its  ease  of  use  and  efficient  operation  bear 
true  v/itness  to  its  untapped  possibilities.  [Ref.  3] 

This  thesis  is  concerned  with  evaluating  the  speed,  accuracy 
and  efficiency  of  the  three  previously  describee  input  devices. 
The   data   gathered   for   analysis   was   generated   by   subjects 
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participating  in  an  experiment  using  the  Naval  Warfare  Interactive 
Simulation  System  (HWISS),  and  will  be  disc us sea  in  detail  in 
later  sections. 

B.   SCOPE  OF  THE  THESIS 

Although  numerous  experiments  concerning  automatic  speec;: 
recognition  have  been  conducted  at  the  Naval  Postgraduate  School 
(NPS)i  Monterey i  California,  only  one  previous  experiment  (conducted 
by  Wright  ana  Hanson;  two  NPS  graduate  students)  has  attempted  a 
comprehensive  evaluation  of  computer  input  technologies  ir  a 
military  command  ana  control  environment.  That  experiment  was  the 
first  attempt  to  evaluate  two  voice  recognition  systems  of  differing 
technologies,  besides  the  more  conventional  keyboard  (typing) 
mode  of  input.  [Ref.  2] 

Accepting  the  conclusions  drawn  in  the  Wright/Hanson  thesis  as 
a  basic  premise,  the  Discrete  Voice  System  (Threshold)  was  eliminated 
as  a  possible  candidate  for  further  experimentation.  This  decision 
was  based  on  its  record  of  poor  performance  in  the  experiment, 
and  the  requirement  of  a  discrete  system  to  speak  in  broken 
phrases  which  is  unnatural  and  cumbersome  to  human  beings  in 
general.  Since  the  keyboard  was  highly  successful  as  an  input 
medium  in  the  previous  experiment,  the  results  were  used  as  a 
third  comparison  against  the  two  newer  mediums  selected  in  this 
experiment.  Because  the  Wright/Manson  experiment  usee  the  same 
NWISS  wargaming  scenarios,  it  was  not  only  unnecessary  to  repeat 
the  procedure,  but  it  allowed  for  more  concentration  on  the 
exploitation  cf  the  possibilities  present  in  the  "newer  media". 
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The  ULsic  purpose  cf  this  experiment  was  to  evaluate  different 
input  interfaces  to  the  NWISS  wargame  scenarios  resident  on  the 
Digital  Equipment  Corporation's  VAX  11/780  computer.  Having 
selectee  the  appropriate  input  media  (i.e.i  the  Macintosh  micro- 
computer with  drop-down  v.;  in  clow  technology,  the  VERBEX  3000  unit 
with  Automatic  Speech  Recognition  capability  and  the  peripheral 
keyboard  attached  to  the  VAX  computer),  it  was  necessary  to 
establish  certain  Measures  of  Effectiveness  (MOE's)  on  which  to 
base  the  ensuing  comparisons.  It  was  decided  that  the  number  of 
commands  entered  during  one  NWISS  wargaming  session  would  be  the 
primary  MOE.  This  coupled  with  the  number  cf  Control_K's  (entry 
errors  made  by  the  user)  entered  would  provide  the  necessary  data 
in  which  to  answer  the  following  questions:  1)  Was  any  system: 
more  effective,  and  or  easier  to  use?;  2)  Was  any  system,  more 
effective  under  the  varied  environmental  conditions?,  and  3)  '-.'as 
a  clear  learning  curve  established  as  a  result  of  more  practice 
with  a  particular  input  device? 

This  thesis  will  not  include  a  description  of  the  construction 
process  used  to  devise  the  wargaming  scenarios,  or  provide  the 
methodology  used  to  build  the  operating  system;  since  they  both 
already  exist.  Additionally,  the  development  of  the  VAX  11/780 
and  the  resident  NWISS  is  already  complete  and  will  not  be  discusseu. 
All  conclusions  from  the  experiment  and  the  author's  recommendations 
for  follow -on  experimentation/study  succeeds  the  analysis  of 
data,  and  will  provide  some  insight  for  the  future  use  cf  this 
burgeoning  technology. 
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C.   SUMMARY 

This  thesis  comprises  a  description  of  the  initial  experiment! 
including  all  procedures  and  background  material  preceding  its 
inception.  A  summary  of  basic  speech  recognition  technology  i 
Macintosh  microcomputer  technology  and  a  review  of  past  experiments 
with  speech  recognition  at  N P S  is  p r o v i d e o  in  Chapters  11  and 

III.  The  design  process  for  this  experiment  (i.e.,  design  of  the 
scenarios,  determination  of  environments,  background  of  the 
subjects,  and  practice/training  sessions)  is  given  in  Chapters 

IV,  V  and  VI.  Analysis  of  the  data  from  the  experiment  is  provided 
in  Chapter  VII.  All  experiment  conclusions  and  the  authors' 
recommendations  for  further  study/applications  follow  the  analysis 
of  data  section. 

Appendices  F  and  G  provide  detailed  guidance  for  follow-on 
experiments  using  Vertex  3GC0  or  Macintosh  microcomputer  technology, 
respectively . 
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II.   SPEECH  TECHNOLOGY 

A.  INTRODUCTION 

The  use  of  Continuous  Speech  Recognition  Systems  has  expanded 
significantly  over  the  past  three  tc  five  years.  After  first 
finding  success  with  discrete  systems  i  ASR  is  now  being  intro- 
duced for  use  in  critical  areas  of  operation.  The  progress  made 
in  recent  years  is  measured  in  many  different  ways.  The  most 
obvious  is  the  performance  and  capabilities  of  the  machine.  Seme 
cf  the  other  criteria,  however,  are  equally  important.  These 
would  include  increased  vocabulary  size,  distinguish,  ing  between 
easily  confusing  words  and  minimizing  the  effects  of  noise  and 
other  interfaces.  This  chapter  will  oiscuss  some  of  the  theory 
behind  Continuous  Speech  Recognition  Systems. 

B.  DEFINITION  OF  TERMS 

The  real  meaning  of  speech  recognition  may  seer.,  intuitive, 
but  its  derivation  and  appropriate  applications  will  be  described 
further  to  preclude  possible  confusion,  and  to  provide  a  common 
base  for  universal  understanding. 

Speech  Recognition  can  be  defined  as  the  process  of  trans- 
forming the  continuous  acoustic  speech  symbol  into  aiscrete 
representations.  These  symbols  may  be  assigned  proper  meanings. 
and  when  comprehendea .  may  be  usee  to  affect  responsive  behavior. 
The  next  major  distinction  is  made  between  Connected  (Continuous) 
Speech,  or  Isolated  (Discrete)  Speech.  Connected  Speech  is  that 
in  which  there  are  no  significant  pauses.  An  example  would  be  a 
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sentence  spoken  at  a  n  o  r  rr.  a  1  rate.  It  is  a  difficult  t  a  s  a  tor 
computers  to  distinguish  between  where  or.e  word  ends,  and  the 
next  word  begins.  Isolated  speech  does  not  have  this  problei.., 
since  significant  pauses  (i.e..  Turn — Lathe--On )  are  often  enough 
to  allow  distinction.  The  "bottom-line"  goal  of  any  computer 
system  is  to  rapidly  ana  accurately  interpret  the  user's  input. 
and  then  select  and  respond  appropriately.  There  is  no  doubt 
that  it  is  easier  to  make  computers  recognize  words  spoken  in 
isolation,  or  at  worst,  spoken  in  broken  phrases.  In  fact,  there 
are  several  examples  of  this  on  the  market  today.  By  the  same 
token,  however,  speakirg  with  isolated  words  or  broken  phrases  is 
difficult  for  humans  to  achieve,  and  hence  not  desirable.  [Ref.  4 
and  5  ] 

C.   CONTINUOUS  SPEECH  RECOGNITION  BACKGROUND 

As  the  cost  of  equipment  aeclines,  Continuous  Voice  Systems 
are  receiving  increased  attention.  Concentrated  research  started 
in  1971  when  the  Advanced  Research  Project  Agency  (DARFA)  of  the 
Depart ment  of  Defense  allocated  $15  million  dollars  tow a re  a 
program  called  Speech  Understanding  Research  (SUR).  It  was  the 
largest  single  project  ever  undertaken  in  speech  recognition,  ana 
succeeded  in  developing  machines  that  were  capable  of  "understanding" 
continuously-spoken  sentences  involving  a  1000-word  vocabulary. 
[Ref.  6]  Although  the  DARPA  project  was  not  the  first  to  research 
continuous  speech  recognition,  it  spawned  the  ideas  of  several 
other  companies  who  followed  suit  shortly  thereafter. 
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The  DARPA  project  came  to  an  eno  in  1976  with  a  long-awaited 
demonstration  of  several  new  systems  that  could  understand  continuous 
spoken  sentences.  Even  though  the  results  were  far  from  perfect, 
they  were  extremely  enlightening.  The  Harpy  System  for  example, 
far  exceeded  its  design  goals  by  correctly  understanding  95%  of 
the  sentences  spoken  by  five  speakers.  It  usee  a  1011-word  vocab- 
ulary* and  a  highly-constrained  gran. mar  cf  sentences  relevant  to 
a  task  concerning  the  retrieval  of  documents  from  the  computer 
memory . 

In  1973.  IBM  developed  a  system  for  automatic  recognition  of 
continuous  speech  that  correctly  recognizee  96%  of  the  7  digit 
numbers,  with  a  per-digit  accuracy  rating  of  better  than  995.  Ir. 
1976,  Texas  Instruments  developed  a  system  successful  in  verifying 
the  speaker's  voice  before  permitting  access  to  the  medium  itself. 
[Ref.  6] 

At  present,  there  is  a  great  desire  to  improve  the  quality  of 
continuous  speech  recognition  technology.  Due  primarily  to  its 
increased  speed,  naturalness  and  mobility,  researchers  have 
recognized  its  importance  in  nany  varied  applications  (i.e.,  Air 
Traffic  Controller  training,  key  word  iaentif ication  in  monitored 
conversations,  command  and  control  environments,  cockpit  con. muni- 
cations,  etc.),  and  the  necessity  to  exploit  all  capabilities. 

D.   TYPES  OF  EQUIPMENT  USED 

The  NWISS  was  the  nucleus  of  this  experiment.  In  addition, 
the  VER3EX  3000  and  the  Macintosh  microcomputer  were  configured 
for  use  as  two  cf  the  experimental  input  mediums,  and  the  peripheral 
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keyboard  attached  to  the  VT1C2  terminal  of  the  VAX  11/780  used  as 
the  third. 

1  .   Continuous  Speech  Recognition  Equipment 

The  VERB  EX  30  0  0  is  currently  the  "state  of  the  art" 
regarding  Continuous  Speech  Recognition  Systems.  It  is  designed 
for  use  in  industrial  and  commercial  applications!  and  works 
well  in  both  high  and  1 o  w  noise  environments  The  system  is 
capable  of  recognizing  natural  continuous  speech  of  infinite 
length,  and  an  output  buffer  of  240  characters  per  recognition 
set  is  its  only  limitation.  [Ref.  7]  With  a  maximum  of  four 
speecn  processing  boards,  the  VERB EX  3000  can  recognize  up  to  360 
different  words  spread  over  as  many  as  20  different  grammars.  A 
finite  limit  to  grammar  size,  based  on  the  total  number  of  words 
and  complexity  of  the  node  transition  network,  is  necessary  to 
allow  the  device  to  remain  "real-time"  in  terms  of  computation 
speed  and  memory  (i.e..  stored  voice  patterns)  requirements.  Thus. 
the  total  application  may  involve  up  to  360  words,  but  at  any 
instant  the  recognizer  is  dealing  with  only  one  grammar  subset. 
[Ref.  8]  The  VERB EX  3000  system  includes  the  Speech  Application 
Development  System  (SPADS).  This  system  allows  the  user  to  run 
applications/software  other  than  those  produced  by  VEKBEX  3000. 
Nevertheless,  any  program  must  be  written  in  the  programming 
language  Pascal  in  order  to  effectively  control  the  application. 
It  is  through  the  use  of  SPADS  that  Lombardo  (an  N PS  graduate 
student)  was  able  to  develop  a  program  in  which  voice  commands 
were  used  in  conjuction  with  the  wargaming  scenarios  used  in  the 
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MWISS.  [Ref.  8  3  Additional  equipn.ent  for  the  VEREEX  3000  includes 
a  small  speech  unit  which  prompts  the  user  for  the  next  wore  or 
phrase.  This  unit  will  echo  the  command  if  subsets  of  phrases  are. 
needed  to  complete  the  command.  The  subjects  in  this  experiment 
useG  a  Shu  re  SH12A  headset  (close-speaking)  microphone  (as  the 
dedicated  input  device),  and  the  Siegler  ADM  31  terminal  to  link 
the  VERBEX  3000  system  with  the  VAX  11/780.  This  particular 
terminal  echos  all  user  inputs  including  commands  and  errors 
identified  as  Control_K's. 
2 .   Laboratory  Equipn.ent 

This  ex  peri nent  required  three  input/output  terminals 
(VT-102  series),  three  status  board  terminals  (VT-100  series)  and 
three  Ramtek  color  graphic  display  terminals.  The  three  input/output 
terminals  were  assigned  as  control,  umpire  anc  master.  Each 
subject  (as  well  as  the  umpire)  had  a  color  graphic  display  to 
view  the  scenario  in  progress.  Status  boaros  for  the  two  subjects 
provided  accurate  combat  information  for  the  respective  forces, 
and  the  umpire  had  access  to  all  information  on  a  special  status 
board  configured  for  that  purpose. 

E.   PREVIOUS  NAVAL  POSTGRADUATE  SCHOOL  STUDIES 

In  the  past  several  years,  there  have  been  a  multitude  of 
experiments  anc  theses  developed  at  the  MPS  to  realize  the  potential 
and  limits  of  Speech  Recogniton.  This  section  will  review  the 
findings  of  a  few  of  these  studies  as  an  attempt  to  summarize  the 
progress  to  date. 
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In  1981  t  re sear en  occurred  which  analyzed  speech  recognition 
as  a  me  tins  tc  improve  speed  and  reliability  in  the  task  of  imagery 
interpretation.  The  purpose  of  the  experiment  was  to  allow  an 
officer  to  devote  more  time  to  concentrate  on  the  object  being 
identified,  and  less  time  imputing  the  information  on  the  keyboard. 
A  discrete  voice  system  (i.e..  Threshold!  T6C0  equipment)  was 
used  to  conduct  the  experiment,  and  the  analyst  fcuno.  that  the 
voice  system  was  approximately  503  faster  than  typed  data  entry. 
As  far  as  "reporting  accuracy",  no  significant  differences  were 
noted.  [Ref.  9]  Concurrently,  in  1981  a  NPS  student  studiec  the 
practicality  of  using  voice  systems  in  a  war  game  environment. 
The  objective  here  was  to  play  the  game  scenario  utilizing  a 
discrete  system  (T600)  and  a  standard  keyboard.  Performance- 
measures  that  were  evaluated  included  time  requirea  for  each 
input,  and  the  number  of  input  errors.  The  game,  entitled  Warfare 
Environmental  Simulation  (WES),  (later  evolved  into  NWISS)  provided 
the  interactive,  real-time  wargame  used  in  this  experiment.  The 
student's  experiment  suggested  that  the  keyboard  entry  was  faster 
and  resulted  in  fewer  errors.  [Ref.  10]  Why  then  was  there  a 
difference  in  outcomes  between  these  first  two  studies?  Apparently, 
the  method  of  measuring  the  errors  was  different  in  the  WE 3 
experiment.  The  total  errors  for  the  voice  entry  included  recog- 
nition and  operator  errors,  while  the  keyboard  entry  totals  only 
included  the  typing  errors.  For  the  imagery  interpretation  exper- 
iment, the  methods  used  to  measure  errors  for  both  input  devices 
was  identical. 
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Several  other  experiments  using  discrete  systems  were  conducted 
with  similar  results.  The  "better"  method  depended  or.  the  designated 
MOE.  the  experience  level  of  the  subjects,  the  extent  of  the 
training,  of  the  subjects,  etc. 

Ir.  1 984  •  another  MPS  student  developed  a  thesis  that  attempted 
to  evaluate  the  use  of  continuous  voice  recognition  technology 
in  a  wargame  environment.  The  student  developed  the  application 
software  to  run  the  NWISS  wargame  using,  the  VERBEX  3000/continuous 
speech  system.  This  software  became  the  foundation  for  the 
Wr ight/Manson  thesis;  the  final  thesis  before  this  experiment. 
The  authors  of  the  thesis,  Wright  and  Hanson,  evaluated  the 
effectiveness  of  three  input  devices  (continuous  speech,  discrete 
speech  and  the  standard  keyboard)  while  running  the  NWISS  wargame. 
The  nonintuitive  conclusion  suggested  by  Wright  and  Manson  was 
that  the  keyboard  players  did  better  overall.  Further  research 
gave  seme  possible  plausibility  for  this  result.  During  low  light 
sessions  almost  all  the  keyboard  subjects  moved  the  keyboard 
panel  closer  to  the  CRT  to  alleviate  a  perceived  handicap.  A  post 
experiment  survey  also  revealed  that  the  keyboard  group  had  the 
highest  typing  skill  level  of  the  three  groups.  Additionally, 
the  discrete  system  fared  well  below  the  other  rrethods  in  the 
majority  of  the  comparisons,  ano  even  though  snorter  training  was 
required,  incorrect  recognition  was  more  likely  to  occur  under- 
varying  levels  of  ambient  noise  and  dissimilar  user  voice  patterns. 

From  the  results  obtained  in  the  Wright/Manson  thesis  it  can 
be  ascertained  that  while  the  keyboard  still  remains  an  alternative 
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for  computer  interface*  the  discrete  system  (however  effective  it 
may  be)  is  not  natural  to  an  individual's  v;ay  of  speaking,  and 
does  not  merit  further  investigation.  Concurrently,  it  is  also 
recognized  that  more  evaluation  of  the  continuous  speech  recognition 
system  is  needed,  and  it  snail  be  discussed  further  in  later 
chapters  of  this  thesis. 

F.   PROBLEMS  OF  SPEECH  RECOGNITION 

With  any  field  of  study  there  are  many  problems  which  tend  to 
hinder  the  progress  of  new  technology;  and  Continuous  Speech 
Technology  is  no  different  in  this  regard.  Nevertheless,  several 
additional  factors  must  be  considered  when  human  behavior  is 
involved.  For  instance:  Can  an  inexperienced  user  operate  a  voice 
system?  Will  the  system  accurately  recognize  an  individual's 
voice  signals?  In  general,  the  bigger  the  system's  vocabulary 
capacity,  the  more  prior  training  will  be  required.  Also,  if  a 
large  number  of  sentences  and  alternative  phrases  need  to  be 
recognized,  additional  training  may  be  necessary.  The  vocabulary 
itself  plays  an  important  role  in  determining  the  systems  ability 
to  recognize  utterances  with  any  degree  of  consistent  accuracy. 
In  other  words,  it  is  much  easier  to  recognize  1C0  distinct 
words,  than  to  recognize  100  "rhyming"  words/phrases  (i.e.,  a 
"light  house"  vs.  a  "lighthouse"). 

Another  difficulty  occurs  when  there  are  "highly  constrained" 
sequences  of  words  (i.e..  stock  numbers,  telephone  numbers,  etc.) 
or  carefully  formatted  phrases  (i.e.,  air  traffic  control  commands! 
fixed-length  commands  with  each  word  selected  from  a  predetermined 
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vocabulary  i  etc.).  These  words  or  phrases  require  a  mere  complete 
analysis  to  insure  the  proper  context  of  each  word.  It  is  also 
extremely  important  that  the  speaker  articulate/annunciate  each 
word  in  the  command. 

One  of  the  biggest  difficulties  encountered  when  dealing  with 
speech  recognition  is  the  unwanted  ability  of  the  microphone  to 
pick  up  outside  noise,  or  sounds  from  the  environment.  While  it 
is  a  fact  that  real  life  conditions  may  encounter  additional 
noises  causing  degradation  in  the  systems  ability  to  accurately 
recognize  words,  the  judicious  use  of  available  testing  facilities 
will  allow  optimum  conditions  on  a  regular  basis.  Another  possible 
solution  is  to  use  "close-talking"  microphones  or  headsets. 
These  particular  devices  are  able  to  differentiate  between  nearby 
sounds  and  sounds  heard  at  a  distance.  [Ref.  6] 

Cost  is  another  very  common  problem  with  this  technology.  For 
Research  and  Development  companies  and  commercial  users,  the 
initial  cost  is  very  high.  At  this  point  in  time,  the  demand  for 
continuous  speech  recognizers  is  not  enough  to  stimulate  the 
significant  number  of  grants  and  contracts  necessary  for*  acequate 
research. 

The  question  that  is  uppermost  in  the  mirds  of  many  commercial 
investors  is  not  whether  the  equipment  is  affordable,  but  is 
it  necessary?  Do  we  really  need  a  continuous  speech  recognizer? 
Some  people  argue  that  "isolated  word  recognizers"  are  adequate 
for  most  jobs.  The  fact  still  remains  there  is  little  doubt  that 
if  improved,  the  sheer  motility  and  flexibility  of  continuous 
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speech  systems  would  easily  surpass  the  cumbersome  operation 
inherent  in  discrete  systems. 
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III.   MACINTOSH  MICROCOMPUTER  TECHNOLOGY 

A.  INTRODUCTION 

Although  the  cccis  against  its  fruitier,  seemed  monumental  in 
1979.  the  Macintosh  microcomputer!  designed  by  a  small  group  of 
inexperienced  engineers  and  programmers*  is  now  recognized  world-wide 
as  a  technological  marvel  in  personal  computers.  Relying  heavily 
on  symbols  and  associated  pictures  on  the  screen,  it  is  designed 
primarily  to  help  people  conquer  computer  phobia  and  oe  as  easy 
to  use  as  a  television  set.  It  is  this  ease  of  use  attribute  that 
made  it  so  attractive  as  a  possible  testbed  interface  device  for 
wargame  simulation.  [Ref.  11] 

This  chapter  will  focus  on  some  of  the  terminology  inherent 
in  microcomputers,  described  the  origin/background  surrounding 
Macintosh' s  innovative  technology  and  its  adaptability  to  experi- 
mental design,  and  lastly,  describe  the  equipment  that  was  used 
in  the  experiment. 

B.  BACKGROUND 

The  Apple  Corporation's  Macintosh  microcomputer  is  built  on 
three  (3)  primary  ideas: 

1)  second-generation  Lisa  technology  (an  expensive  desktop 
con.puter  designed  for  commercial  use  that  pioneered  the 
merging  of  graphics  and  "mouse "-oriented  peripheral 
operation  )  , 

2)  reliability  and  low  cost  through  inbred  simplicity, 
and 

3)  a  maximumization  of  the  synergism  possible  between  hardware 
and  software. 
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Powered  by  the  dedicated  ambition  cf  Burrell  Smith  (a  former 
Apple  Computer  Corporation  maintenance  technician)  Steven  Jobs 
(the  chairman  of  Apple,  and  the  Macintosh  project  director)  and 
an  elite  project  team  of  "visionary  r a d i c a 1 s "  ,  Apple  has  succeeded 
in  cresting  a  machine  that  weighs  only  20  pounds*  can  be  carried 
in  a  tcte  bag  and  has  many  of  the  significant  features  introduced 
in  January  of  1963  with  the  Lisa  computer.  Although  it  uses  a 
"mouse"  (a  pointing  device  roughly  the  size  of  a  pact:  of  cigarettes) 
that  permits  users  to  give  commands  to  the  computer  via  arm/hand 
movements  and  button  pushing,  there  are  four  (4)  basic  differences 
from  the  Lisa  that  make  it  second-generaticn  oriented: 

1)  a  much  higher  clock  speec  (7.83  MHZ  vs.  the  Lisa's  5.0 
MHZ  )  , 

2)  smaller  memory  used  more  efficiently  through  the  MC68000 
processor . 

3)  an  elimination  of  add-on  peripheral  cards  through  the 
extended  use  of  a  high  speed  serial  bus  coupled  with  the 
implementation  of  "virtual  slots",  and 

4)  it  allows  only  one  applications  program  to  be  active  at  a 
time  (an  important  limitation). 

Although  this  latter  limitation  aspect  seems  serious  in 

nature,  it  actually  facilitates  the  use  of  multiple  window  operations 

and  allows  material  to  be  "cut  and  pasted"  from  one  document  to 

another  by  a  "clipboard"  arrangement.  Nevertheless,  the  absence 

of  hardware  slots  and  the  lack  of  ability  to  run  two  simultaneous 

applications  are  the  two  important  separation  criteria  from  that 

of  the  Lisa  technology.  [Ref.  12] 
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By  combining  the  functions  of  many  stanaard  chips  into  eight 
programmable  logic  arrays,  it  makes  it  possible  for  the  Macintosh 
to  keep  its  chip  count  (and  therefore  its  price)  relatively  low, 
while  simultaneously  producing  a  machine  that  has  a  reliability 
factor  characteristically  associated  with  much  sit.  pier  computers. 
With  only  two  printed  circuit  boaras,  it  was  cleverly  designed  to 
eliminate  the  need  for  hardware  fire-tuning  during  assembly  or 
use.  It  is  tnis  combination  of  reliability  ana  simplicity  that 
makes  it  an  ideal  training  or  input  interface. 

The  Macintosh's  hardware  and  software  are  fully  optimized  for 
maximum  potential.  This  simply  means  that  the  hardware  and  software 
have  evolved  slowly  ever  a  period  of  time  that  has  prcvioec  a 
mutual  "give  and  take"  association.  As  a  result,  the  video  display 
hardware  design  greatly  simplifies  the  software  design;  allowing 
it  to  easily  draw  squares  and  circles,  scale  text  and  graphics 
and  print  screen  images.  Utilizing  this  feature  to  its  fullest, 
Tom  Concannon  (a  MPS  graduate  student)  was  able  to  develop  the 
MWISS  wargame  interface  program  in  the  BASIC  language  to  allow 
the  experiment  subjects  to  play  the  various  wargame  scenarios. 
This  program  activated  the  "user  toolbox",  which  is  primarily 
responsible  for  manipulating  all  the  various  relative  relationships 
between  different  units/accessories  in  the  MC68000  processor.  By 
stimulating  the  window  and  menu  managers,  the  subjects  were  then 
able  to  use  the  mechanical  mouse  to  select  wargame  comnanas  from 
a  two-dimensional  matrix  of  menu  items.  The  selected  input  was 
coordinated  through  a  resource  manager  facility  which  perpetuated 
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the  eventual  interface  necessary  for  communication  and  coir.ti.ara 
actuation  on  the  VAX  11/780  computer.  An  example  of  each  menu 
column  followed  by  all  the  possible  user  selections  is  contained 
in  the  pictorial  section  of  the  Macintosh  Microcomputer  User's 
Guide  (see  Appendix  G ) .  [ R e f .  3] 

C.   EXPERIMENT  HARDWARE  DESCRIPTION 

The  Macintosh  has  received  numerous  accolades  for  its  impressive 
design.  It  is  truly  a  highly  developed  microcomputer  that  effectively 
uses  a  mechanical  mouse  peripheral  and  a  32  bit  microprocessor  in 
a  lightweight  portable  package.  It  is  its  portability,  functional 
simplicity  and  ease  of  use  that  make  it  the  ideal  selection  for  a 
user  interface  device. 

The  Macintosh  microcomputer  is  a  self-contained  unit  (9.75  by 
9.75  by  13.5  inches)  consisting  of  a  main  unit,  keyboard  and 
mechanical  mouse.  It  uses  standard  AC  external  power,  has  two 
serial  ports  and  an  integral  Sony  3.5  inch  disk  drive  which  holds 
up  to  400  Kbytes  of  information.  Its  standard  128  Kbytes  of  RAM, 
and  64  Kbytes  of  ROM  were  reconfigured  before  the  actual  experiment 
to  512  Kbytes  to  accomodate  the  slight  command  entry  delay  experi- 
enced by  users  at  the  lower  configuration  setting.  The  nine  (9) 
inch  monochrome  monitor  provided  an  adequate  display  of  all 
pictorials,  and  its  58  key  detachable  keyboard  served  as  a  comfor- 
table medium  for  a  backup  typing  input  mode.  The  main  source  of 
user  input  was  a  small  mechanical  mouse  which  is  physically  moved 
in  a  rectangular  series  of  hand  movements  to  position  an  onscreen 
arrow  to  the  desired  menu.  Keeping  the  index  finger  positioned  on 
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a  conveniently  mounted  button,  the  user  manipulated  the  mouse  to 
position  the  arrow,  and  used  the  button  to  extena  the  desired 
menu  and  select  the  appropriate  command  for  subsequent  implemen- 
tation. A  detailed  description  of  the  mechanics  involved  in  user 
operation  is  again  available  by  referencing  Appendix  G .  [ R e f s .  12 
and  3] 

RAMTEK  high  resolution  color  graphics  monitors  were  used  to 
display  the  game  scenarios  to  the  subjects  at  their  respective 
positions,  and  the  digital  VT1G0  terminal  was  used  as  a  status 
board  for  each  NWISS  session.  The  configuration  of  the  laboratory 
itself  will  be  further  described  in  Chapter  VI,  anc  a  pictorial 
display  of  the  physical  layout  provided  (see  Figure  6.1). 

A  conventional  cassette  tape  recorder  was  used  to  play  a 
pre-recorded  background  noise  tape  uuring  each  NWISS  session.  One 
eight  inch,  eight  ohm  speaker  was  placed  at  a  pre-established 
distance  from  each  subject,  and  an  audiometer  was  used  to  verify 
the  decibel  settings  for  each  light/noise  environment  specified. 
The  pre-recorded  noise  consisted  of  a  gathering  of  approximately 
thirty  (30)  people  speaking  in  a  conversational  tone  within  an 
auditorium.  [Ref.  2] 

D.   SUMMARY 

This  chapter  has  served  to  discuss  some  of  the  terminology 
inherent  in  the  design/description  of  the  Macintosh  microcomputer, 
and  has  presented  some  of  the  background  theory  and  history 
behind  its  origin.  Although  there  were  no  previous  wargame  scenario 
experiments  conducted  at  the  MPS  involving  the  use  of  the  Macintosh 
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(or  similar  technology)  as  an  input  medium,  its  ease  of  use,  and 
its  overall  potential  prompted  its  employment  in  this  experiment. 
Finally,  a  description  of  the  actual  microcomputer  ha  r  civ;  a  re  and 
associated  laboratory  equipment  included  in  the  experiment  was 
provided. 
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IV .   THE  NAVAL  WARFARE  INTERACTIVE  SIMULATION  SYSTEM  (NWISS) 

A.  INTRODUCTION 

This  chapter  contains  a  brief  description  of  NWISS  arc:  its 
support  role  in  this  experiment.   All  scenarios  developed  for  the 

experiment,  the  measures  oV  effectiveness  selectee  ana  the  methods 
usee  to  collect  data  and  compare  the  two  command  input  technologies 
are  also  described. 

B.  WHY  NWISS  WAS  SELECTED 

NWISS  is  a  real-time,  user-interactive  simulation  of  r.aval 
warfare.  Its  mission  'was  originally  to  train  senior  Naval  Officers 
in  force-level  tactical  decision  making  and  management  of  cor.:. sane 
and  control.  At  the  Naval  Postgraduate  School,  NWISS  resides  en 
a  VAX  11/780  computer,  and  the  peripheral  VT1C0/102  terminals  are 
P.  A  K  T  E  K  graphics  terminals  provide  the  necessary  input/output 
modules  when  grouped  together.  Voice  equipment  (i.e.,  the  VEREEX 
3000),  ane  "drop-down"  window  microcomputer  technology  (i.e.,  the 
Macintosh  microcomputer)  were  used  as  interface  devices  in  the 
available  modules,  and  all  inputs  and  outputs  were  accomplished 
by  trained  subjects.  NWISS  was  loaded,  controlled  and  monitored 
from  a  thire  "command"  module  during  these  experiments.  The 
purpose  of  NWISS  in  this  experiment  was  to  stimulate  a  stressful 
environment  that  would  allow  user  intervention  via  NWISS  commands. 
From  the  user's  point-of-view,  it  has  a  large  vocabulary  of 
commands  to  change  the  view  of  the  tactical  situation  in  the 
battle  arena,  cause  changes  in  the  course  and  speed  of  vehicles 
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in  one  forcei  ana  launch  air  assets  with  user  defined  weapon 
loads  and  missions.  Nl/ISS  can  also  show  the  status  of  the  v:\er'  s 
forces  on  several  menu  selectea  displays.  For  this  experiment! 
each  station  cons  is  tea  of  a  cor,  nana  input  terminal ,  a  force 
status  terminal  and  a  video  monitor  to  graphically  display  force- 
disposition  and  identity.  [Ref.  2 ] 

Nl/ISS  was  used  to  support  this  experiment  because  it: 

1.  OffereO  the  ability  to  present  identical  tactical  situations 
to  two  isolated  subjects  simultaneously.  This  controlled 
environment  allowed  a  side-by-side  comparison  of  the  command 
input  methods  chosen  by  the  isolatea  subjects  when  confronted 
by  identical  tactical  stimuli.  Additionally,  the  system 
was  readily  available  at  the  MPS,  and  a  pool  of  trained 
operators  were  accessible  for  "on-line"  consultation. 

2.  Provides  a  large  system  vocabulary.  Consensus  of  opinion 
was  that  if  a  small  vocabulary  was  chosen  to  run  the 
experiment,  the  subjects  may  become  bored  through  sneer 
command  repetition.  Approximately  150  words/utterances  were 
deemed  necessary  to  properly  conauct  the  variety  of 
inherent  missions  with: in  the  capabilities  of  an  Aircraft 
Carrier  Battle  Group.  Twenty  commands  relating  directly 
to  the  control  of  the  tactical  force  display  were  also 
included.  Tables  of  the  complete  vocabularies  for  the 
two  input  technologies  are  included  in  Appendices  F  &  G. 
These  vocabularies  were  predetermined  and  were  limited  by 
the  size  of  the  memory  of  the  VERBEX  3000  system.  Using 
the  forces  and  commands  in  these  subsets  of  the  Nl/ISS 
dictionary,  four  previously  developed  scenarios  were  used 
as  a  medium  for  stimulating  subject  irterest,  ana  for 
generating  a  meaningful  MOE.  [Ref.  14] 

In  all  four  scenarios,  the  Orarge  and  Neutral  force  actions, 
defensive  posture  and  radar  emissions  control  (EMCON)  status  were 
preprogrammed.  Orange  aircraft  were  eliminated  from  this  experiment, 
to  avoid  unnecessary  system  program  complexity.  The  two  available 
views  of  the  world  for  the  Blue  forces  were  simulated  on  approxi- 
mately the  same  latitude,  but  were  physically  separated  in  longitude 
by  500  nautical  miles  at  mid-ocean. 
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h'WISS  is  limited  to  a  maximum  of  six  sets  of  forces,  and  the 
Grange  and  Neutral  forces  were  each  structured  appropriately  to 
reflect  one  v  i  e v; .  The  specific  process  for  assembling  these 
various  views  and  overlapping  them  is  explained  in  some  cetai]  in 
a  previous  MPS  thesis  by  Owens  ana  Drown.  [P.ef.  15]  Basically, 
there  were  two  separate  views  for  the  Blue  forces  which  allowed 
the  subjects  to  individually  control  their  own  ships  ana  planes. 

C.   SCENARIOS 

Four  unique  scenarios  were  used  in  order  to  effectively  place 
subjects  in  situations  requiring  the  use  of  the  available  KV/ISS 
vocabulary,  and  to  stimulate  individual/different  responses.  This 
section  describes  the  basic  opening  force  positions  for  each  of 
the  scenarios  and  the  accompanying  military  situation  briefing 
for  that  scenario.  The  actual  situation  briefings  giver:  to  the 
subjects  are  included  in  Appendix  A. 

1 .   Scenario  A 

One  aircraft  carrier,  twelve  P3C  aircraft,  two  frigates, 
and  an  attack  submarine  are  assigned  to  the  Blue  forces.  Orange 
forces  include  one  aircraft  carrier  and  six  surface  combatants* 
and  are  escorting  eight  neutral  merchant  ships  in  a  convoy  that 
is  moving  south  as  shown  in  Figure  4.1.  The  merchant  ships  are 
reportedly  carrying  nuclear  and  chemical  weapons  to  an  unidentified 
Orange  ally.  Hostilities  have  been  declared  between  trie  Blue 
and  the  Orange  forces  and  the  objective  is  to  identify  and  destroy 
the  Orange  escort  vessels  without  involving/destroying  the  merchant 
ships. 
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2 .  Scenario  E 

Blue  fcrces  consist  of  two  P3C  aircraft  to  provide  the 
necessary  targeting  i  nf  or  ma  t  i  on ,  one  aircraft  carrier  anc  two- 
flights  of  five  attack  aircraft.  Four  Grange  surface  combatants 
are  positioned  between  these  Blue  forces,  ana  twenty  merchant 
vessels  are  at  anchor  outside  a  fictitious  harbor  facility  waiting 
to  unload  their  cargo.  Hostilities  have  not  been  declared  between 
Blue  and  Orange  forces,  although  tensions  remain  righ.  The  Blue 
force's  mission  is  to  use  the  assigned  airborne  aircraft,  anc  the 
remaining  aircraft  carrier  based  assets  to  conduct  a  strike  on 
the  merchant  vessels.  Rules  of  engagement  do  not  permit  firing  by 
the  Blue  forces  until  the  Orange  forces  have  fired  on  them.  From 
the  opening  positions  as  shown  in  Figure  4.2,  neither  the  merchant 
ships  nor  the  Orange  combatants  are  visible  to  the  Blue  forces. 
but  a  northerly  course  is  dictated  by  the  mission. 

3 .  Scenario  C 

A  favorable  environment  is  present  in  which  Blue  tasking 
is  to  locate  and  identify  as  many  objects  as  possible  and  designate 
them  as  Friendly.  Neutral  or  Enemy.  Blue  forces  have  one  P3C  ana 
four  F14A  aircraft  to  accomplish  this  mission,  and  Combatants 
for  both  Blue  and  Orange  forces,  as  well  as  several  merchant 
ships  and  fishing  vessels,  are  all  in  a  400  X  500  nautical  mile 
rectangle  (as  depicted  in  Fig.  4.3).  No  control  of  the  Blue 
surface  forces  was  permitted  to  complicate  the  tasks  of  location 
and  identification. 
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Two  carrier  battle  groups  of  approximately  equal  strength 
are  pitted  against  each  other.   Slue  forces  are  specifically 

tasked  to  locate  and  identify  the  elements  of  the  Orange  force 
battle  group.  Hostilities  nave  not  been  oeclarec.  but  the  subjects 
were  allowed  v/ide  latitude  in  their  response  to  any  Orange  force 
aggressive  action.  The  opening  positions  in  Figure  4.4  place  the 
Orange  force  carrier  battle  group  to  the  North  cf  the  Blue  force- 
group;  with  the  Orange  force  headed  North  and  trie  Blue  force 
headed  to  the  Northwest. 

D.   MEASURES  OF  EFFECTIVENESS 

All  subjects  were  familiar  with  the  NWISS  wargaming  scenarios. 
Some  of  the  commands  (i.e..  changing  the  scale  of  the  display) 
were  easy  to  accomplish  ancl  rapidly  assimilated,  while  others 
(such  as  the  launch  sequence)  required  patience  ana  considerable 
practice  to  implement  correctly.  When  the  subjects  receivec  their 
mission  briefing  before  each  session,  they  were  told  some  of  the 
measures  of  effectiveness  by  which  treir  success  or  failure  would 
be  determined.  Included  were  such  tilings  as  the  number  of  merchant 
vessels  sunk  or  damaged  in  Scenario  B,  the  number  of  ships  correctly 
located  and  identified  in  Scenario  C,  and  the  number  of  Blue 
forces  lost  to  hostile  fire  in  Scenario  A  (sec  Appendix  A).  In 
each  case,  the  subject  was  urged  to  accomplish  the  g°al  within 
the  allotted  time  period  (i.e..  40  minutes).  These  MOE's  were 
false  sets  of  FiOE's,  and  although  not  used  for  analysis,  were 
intended  to  stimulate  the  subjects  to  achieve  a  meaningful  ^oal. 
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Figure  4.1    Scenario  A 
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Figure  4.2    Scenario  B 
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Figure  4.3    Scenario  C 
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Figure  4.4   Scenario  D 
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The  actual  measures  of  effectiveness  are  related  tc  the 
number  of  KUISS  commands  entered  and  their  corresponding  accuracy. 
The  number  of  commands  entered  was  considered  to  be  t h e  most 
precise  inGicator  of  subject  mastery  over  the  input  technique.  It 
was  felt  that  the  four  scenarios,  with  the  applicable  combinations 
of  light  and  noise  levels,  provided  the  opportunity  for  the 
subject  to  use  commands  available  under  varying  environmental 
conditions.  Real-world  time  constraints  and  accompanying  pressures 
were  impressed  on  the  subjects  by  the  schedule,  ana  by  the  delib- 
erate, overt  actions  of  the  facilitators.  Since  all  the  session s 
lasted  the  same  number  of  game  minutes,  the  number  of  commands 
entered  was  essentially  a  rate  of  entry.  The  astute  subject  could 
have  increased  this  rate  significantly  simply  by  repeatedly 
varying  trie  display  racius.  by  stressing  the  fictitiously  generated 
HGE's,  this  type  cf  performance  was  indirectly  discouraged. 

It  was  determined  through  supervisor  consultation  and  group 
advisory  discussion  that  accuracy  can  be  measured  to  some  extent 
by  the  number  of  times  the  "Control_X"  command  is  used  to  celete 
an  attempted  command.  Nevertheless,  there  are  some  limitations 
to  using  "Control_K"  as  a  measure  of  the  number  of  errors  made  by 
subjects.  Since  it  aborts  the  current  command  and  does  not  execute 
it,  the  "Control_K"  coulc  be  viewed  as  a  change  of  mind  on  the 
subject's  part.  From  prior  experience  with  the  NWISS  game  it  is 
as  easy  tc  enter  a  completed  command  and  issue  a  subsequent 
correction,  as  it  is  to  abort  the  command  completely.  [Ref.  2] 
There  are  also  two  equipment  related  factors  that  detract  from 
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the  use  of  "Control_K"  as  an  error  counter.  First,  in  the  VEREEX 
3000  speech  recognition  system,  alter  the  keyboard  has  beer  used 
to  enter  an  KVJISS  command  not  included  in  tne  155  wore  vocabulary  • 
the  "Control_K"  command  is  necessary  to  return  to  the  beginning 
o:"  the  command  input  sequence.  Therefore,  this  use  of  "Control__K" 
does  not  indicate  an  error,  out  tne  correct  use  of  the  system. 
Secondly,  it  could  be  that  the  subject's  utterance/wore  was  not 
recognized  by  the  system  anc  the  subject  elected  to  enter  the 
command  manually.  The  result  is  the  same  as  in  the  first  case: 
the  abort  command  must  be  used  to  return  to  the  start  of  a  new 
command  input  sequence.  In  a  Deliberate  attempt  to  alleviate  this 
particular  "discrepancy",  the  authors'  evaluated  the  printout  of 
the  sessions  immediately  following  their  execution  so  that  memory 
of  the  subject's  rationale  behind  the  selection  of  a  Cor.  trcl_K 
input  could  be  interjected,  and  subsequently  discarded  if  it  was 
not  intended  as  an  error  correction.  Overall,  despite  these 
considerations,  it  was  felt  that  the  "Contrcl_K"  command  provided 
the  best  measure  available  of  errors  made  in  entering  commands; 
although  it  still  may  not  be  as  strong  a  measure  as  the  number  of 
commands  entered. 

E.   DATA  COLLECTION 

Data  collection  was  accomplished  through  several  utility 
programs  of  the  VAX  11/780.  All  commands  en  t  ere  a  by  the  two 
subject's  command  terminals  were  sent  to  separate  files  in  memory . 
as  was  the  record  of  the  umpire's  game  control  terminal.  This 
process  provided  a  record  of  all  subject  activity,  anc.  kept  a 
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historical  account  of  all  engagements  .  weapon  firings  and  resultant 
damage  displayed  on  the  umpire's  terminal  for  further  study.  The 
output  was  then  hand-counted  to  determine  the  number  of  commands 
entered  and  the  number  of  errors  committed. 

One  additional  comment  is  warranted  regarding  how  the  number 
of  commands  entered  was  actually  determined.  The  aircraft  launch 
sequence  is  widely  used,  and  is  the  most  lengthy  and  difficult 
command.  Since  the  aircraft  launch  sequence  involves  at  least 
five  lines  to  be  complete,  and  could  potentially  be  aborted 
anywhere  within  those  lines,  each  line  of  the  five  that  was 
correctly  entered  was  counted  as  a  command  entered. 
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V.  EXPERIMENT  BACKGROUND 

A.  INTRODUCTION 

Several  independent  factors  were . considered  crucial  to  the 
success  of  this  experiment.  First  was  the  continued  operational 
status  of  tne  critical  "hardware"  (i.e.*  the  VAX  computer,  the 
NWISS  warganiei  the  VERBEX  3000  system  and  the  Macintosh  computer). 
Secondly,  a  total  of  24  subjects  were  needed  to  participate  in 
the  experiment .  Decisions  such  as  the  length  of  each  session,  the 
randomness  of  the  sequences  and  the  combinations  of  scenarios  and 
environments  were  also  of  major  importance.  This  chapter  will 
describe  the  actual  training  and  practice  sessions  using  the 
VERBEX  3000/ Macintosh  microcomputer  technology,  aiscuss  the 
background  of  the  subjects,  and  elaborate  on  the  details  surrounding 
the  design  of  the  experiment. 

B.  BACKGROUND  OF  SUBJECTS 

The  24  subjects  who  participated  in  the  experiment  had  a 
rryriad  of  backgrounds.  Twenty-three  of  them  were  armed  forces 
officers  (i.e.,  U.S.  Navy,  U.S.  Army*  U.S.  Air  Force,  etc.),  and 
all  were  studying  Command,  Control  and  Communications  (C3)  at  the 
Naval  Postgraduate  School  (NPS).  Nineteen  were  male  and  four  were 
female.  The  twenty-fourth  subject  was  a  male  civilian  computer 
operator  employed  at  MPS. 

All  the  students  participated  in  the  experiment  in  conjunction 
with  OS  4602,  a  required  Operations  Research  course  titled  Command. 
Control  and  Communications  Systems  Evaluation.  A  previous  Operations 
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Research  course,  OS  3603  (Simulation  and  bargaining),  introduced 
the  students  to  the  NWISS  background  and  syr.bclogy  used  for  the 
experiment,  and  a  similar  course,  OS  3404  (Man-Machine  Interaction)  , 
introduced  them  to  Discrete  Automatic  Voice  Recognition  Theory. 
All  the  subjects  were  familiar  with  the  basic  facilities;  that 
is,  the  digital  VT100/102  keyboard  terminals  and  the  Ramtek 
graphics  monitors.  Mone  of  the  subjects  had  ever  used  a  continuous 
automatic  speech  recognition  system,  and  some  had  never  used  the 
particular  headset-type  microphone  incorporated  by  the  VERBEX 
3000  unit.  Additionally,  out  of  all  the  Macintosh  players,  only 
five  had  ever  used  a  mouse,  and  due  to  the  specific  nature  of  the 
software  program,  it  woulc  be  the  first  time  anyone  had  played  the 
HWISS  game  using  that  device.  The  civilian  programmer  v/as  to  be  a 
VERBEX  3000  player,  and  although  he  was  familiar  with  the  NWISS 
game,  he  had  never  played  it  using  the  continuous  voice  interface. 
The  24  subjects  were  divided  into  two  equal  teams  of  12 
people.  Each  team  would  only  use  one  input  device  (either  the 
VERBEX  3000  or  the  Macintosh).  Selection  was  mace  randomly,  and 
on  that  basis  a  VERBEX  3000  player  and  a  Macintosh  player  were 
partnered  so  they  could  be  scheduled  for  each  session  simultane- 
ously. Similar  classroom  schedules  v/ere  the  only  matching  prereq- 
uisite, and  although  the  amount  of  exposure  to  the  NWISS  was 
equal  for  all  subjects,  a  similar  statement  could  not  be  made 
concerning  each  individuals  competence  v/ith  the  same. 
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C.   ENVIRONMENTAL  CONDITIONS 

The  experiment  was  constructed  to  simulate  a  military  command 
and  control  center.  To  accurately  represent  this,  four  different 
environmental  conditions  v;ere  used.  One  of  four  combinations  of 
noise  and  light  levels  was  established  for  each  run  of  the  exper- 
iment, and  one  of  four  different  scenarios  was  used  to  allow  each 
player  to  experience  a  different  set  of  circumstances  and  environ- 
mental conditions  for  each  individual's  wargaming  sessions. 
Additionally,  by  alternating  scenarios  it  helped  to  preclude 
familiarity,  and  any  possible  unfair  advantage  achieved  through 
intimate  knowledge  of  one  scenario  in  particular. 

Noise  and  light  conditions  used  for  the  experiment  are  summarily 
described  as  follows:  low  noise  (65  db)  ,  high  noise  (85  db)  , 
normal  lighting  (approximately  1  f oot-lambert )  and  low  lighting 
(lights  completely  out;  approximately  .05  f oot-lambert)  .  The 
introduced  noise  was  measured  in  decibels  with  an  audiometer 
(on  the  C-scale) ,  and  readings  were  taken  at  the  subject's  "ear- 
level"  with  the  stereo  speaker  approximately  2-3  feet  away. 
Light  conditions  were  measured  with  a  light  meter.  normal  light 
conditions  of  1  foot-lambert  were  achieved  by  maintaining  the 
overhead  track  lights  on  and  directed  at  a  45  degree  angle  tov/ard 
the  nearest  wall.  With  the  lights  out  in  the  lev;  lighting  setting 
the  only  lighting  available  to  the  subject  was  the  light  emitted 
from  the  associated  CRT  and  Ramtek  graphics  screen. 

To  the  greatest  extent  practicable,  physical  partitions  were 
used  to  maintain  the  appropriate  noise/light  conditions  throughout 
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each  session.  Full  cooperation  was  experienced  from  the  other 
users  of  the  C3  wargair.ing  laboratory,  and  a  stable,  controlled 
experimental  atmosphere  maintained  (see  Figure  6.1,  Chapter  VI, 
for  a  complete  Laboratory  Layout) . 

D.   SCHEDULING  OF  SESSIONS 

The  NWISS  experiment  took  place  in  the  C3  wargaming  laboratory 
at  the  Naval  Postgraduate  School.  It  was  intended  to  be  run  in 
lieu  of  the  subjects  scheduled  laboratory  period  for  their  OS 
3603  course.  The  subjects  would  participate  individually  in  four 
sessions;  each  run  was  scheduled  to  last  for  40  minutes.  A  short 
brief/debrief  period  v/ould  make  the  entire  session  approximately 
50  minutes  in  total  length. 

The  experiment  was  conducted  from  15  October  85  through  28 
October  85,  and  during  that  period  a  total  of  96  sessions  v/ere 
recorded.  The  C3  laboratory  proved  to  be  an  ideal  location  for 
the  experiment.  Because  the  NWISS  game  is  resident  on  the  VAX 
11/780,  and  the  VAX,  the  VERBEX  3000  and  the  Macintosh  computers 
v/ere  all  located  in  the  same  laboratory,  they  were  all  easily 
configured  and  able  to  communicate  v/ell  with  each  other. 

The  experiment  was  conducted  early  in  the  Fall  academic 
quarter  to  alleviate  traffic  congestion  throughout  the  laboratory 
by  non-participants.  Since  the  laboratory  was  divided  into  three 
equal  bays,  each  player  was  completely  separated  from  the  other 
player,  and  in  almost  total  isolation  from  other  laboratory  users. 

Actual  scheduling  was  done  on  a  random  basis,  and  changes 
were  only  made  if  conflicts  arose  between  team  members.  The 
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cooperation  exhibited  by  the  subjects  in  this  regard  v/as  greatly 
appreciated  as  only  a  few  last  minute  changes  had  to  be  race.  See 
Appendix  E  for  a  complete  listing  of  subjects,  and  the  actual 
experiment  schedule  of  sessions.  [Ref.  16] 

E.  SCENARIO/ENVIRONMENT  SEQUENCE 

A  separate  scenario  and  environmental  condition  was  assignee 
for  each  session,  and  each  team  played  the  NWISS  under  all  four 
combinations.  For  example,  Team  4  played  the  NWISS  using  Scenario 
A  and  Condition  4,  Scenario  B  and  Condition  3,  Scenario  C  and 
Condition  2,  and  finally  Scenario  D  and  Condition  1.  The  order  of 
assigning  scenarios  was  counterbalanced  for  sequential  positioning, 
and  for  both  preceoing  and  following  treatment  effects.  [Ref. 
17]   (See  KEY  in  Appendix  E.) 

The  assigning  of  environmental  conditions  was  also  adjusted 
using  the  same  methodology.  The  schedule  was  adhered  to  as  closely 
as  possible,  and  even  v/ith  three  separate  last  minute  session 
changes,  all  subjects  played  each  scenario  and  environmental 
condition  once. 

F.  CONSTRAINTS 

Due  to  its  secure  environment,  and  the  fact  that  all  the 

wargaming  scenarios  v/ere  already  available  on  the  VAX  computer, 

the  C3  laboratory  was  the  perfect  place  for  running  military 

wargames.  Nevertheless,  certain  features  inherent  to  the  laboratory 

itself  led  to  the  following  constraints  during  the  experiment: 

1.   NUTSS  is  the  only  wargame  available  at  the  NPS  that  has 
the  software  available  to  allow  the  use  of  the  VEREEX 
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3000  equipment  for  the  input  of  voice  commands.  The 
experiment  was  therefore  constrained  to  using  the  VERBEX 
3000  system  for  the  continuous  voice  input. 

2.  The  physical  layout  of  the  laboratory  itself  restricted 
the  placement  of  the  VERBEX  3000  and  Macintosh  input 
devices  to  keep  a  lighted  secticn  of  the  laboratory  open 
for  other  student  use. 

3.  The  versatility  of  the  laboratory  allows  it  to  be  usee 
for  a  myriad  of  student/instructor  projects.  Because  of 
this  fact  the  experiment  had  to  be  conducted  at  a  time 
when  the  laboratory  could  be  exclusively  reserved,  to 
prevent  interference  with  the  established  environmental 
conditions.  As  a  result,  it  was  necessary  to  complete 
the  experiment  in  October.  Additionally,  the  sessions 
had  to  be  designed  to  coincide  with  the  regularly  scheculed 
laboratory  periods  of  the  OS  4602  students  in  order  not 
to  conflict  with  their  individual  schedules.  The  combination 
of  all  these  particular  constraints  prudently  limited  the 
actual  sessions  to  four  per  subject.  Each  session  lasted 
approximately  50  minutes  to  allow  sufficient  time  to 
properly  brief  the  subjects  and  collect  pertinent  data. 

4.  The  subjects  were  randomly  selected,  and  permanently 
assigned  to  only  one  mode  of  command  input  due  to  the 
previously  mentioned  time  constraints,  as  well  as  to 
increase  their  personal  proficiency  with  the  assigned 
input  device. 

G.   TRAINING 

1.   General 

As  delineated  earlier,  all  students  were  assigned  to 
either  the  continuous  voice  recognition  system  (VERBEX  3000)  or 
the  Macintosh  microcomputer  located  in  the  C3  Wargaming  Laboratory. 
Each  individual  was  given  a  thorough  briefing  on  how  to  use  the 
appropriate  device,  and  how  to  train  the  equipment  to  either 
recognize  their  unique  utterances  of  the  designated  NfaISS  commands 
(for  the  voice) ,  or  enter  the  necessary  commands  selected  from 
the  appropriate  "drop-down"  display  window  (for  the  Macintosh) . 
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2 .  Environment  Purine  Training  Sessions 

Normal  laboratory  lighting  conditions  were  allowed  through- 
out the  training  sessions.  This  normal  level  of  one  to  two  foot- 
lamberts  would  be  employed  during  two  of  the  experiment's  four 
simulated  environmental  conditions,  and  simultaneously  facilitated 
NWISS  phraseology  reference  for  the  user's  ref amiliar ization/ 
retention. 

A  cassette  tape  supplied  continuous  background  noise  at 
a  level  of  sixty-five  decibels.  A  previous  vcice  experiment  had 
determined  a  direct  correlation  between  the  consistent  use  of 
this  decibel  noise  setting  and  the  best  possible  climate  for 
voice  recognition.  [Ref.  18]  Besides  its  acceptance  as  the  best 
noise  level  for  recognition  accuracy,  it  was  discovered  that  the 
background  noise  with  the  presence  of  a  fixed  air  vent  recirculation 
fan  would  provide  the  minimum  level  of  noise  present  throughout 
the  course  of  the  experiment. 

3.  VERBEX  3000  Voice  Training 

To  allow  the  VERBEX  3000  system  to  react  properly  to  the 
150  preprogrammed  NV.ISS  command  words,  it  was  necessary  for  all 
designated  voice  subjects  to  engage  in  tv/o  specific  phases  of 
voice  recognition  conditioning.  The  first  phase  requirea  the 
consistent  repetitious  uttering  of  all  150  words  individually  in 
order  to  allow  the  VEPBEX  3000  unit  to  logically  associate  each 
word  with  each  users  unique  voice  characteristics,  and  then 
physically  enroll  this  "transformed"  version  of  the  v/ord  into  its 
memory  for  later  recall/recognition.  This  procedure  took  an 
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average  of  40  minutes  for  each  user,  and  although  a  standardized 
pronounciat ion  of  the  word  was  supplied  and  encouraged  ,  trie 
subjects  v/ere  free  to  substitute  any  utterance  they  desired  for 
the  VERBEX  3000  prompt  (if  it  was  remembered  and  used  curing  the 
actual  wargaming  sessions).  In  the  second  phase,  a  maximum  of 
three  of  the  150  possible  utterances  v/ere  concatenated,  and  then 
grouped  by  similar  usage  into  nine  specific  grammars  or  sections. 
Each  partial  command  phrase  v/as  then  repeated  continuously,  and 
as  consistently  as  possible  (using  voice  inflection  and  volume 
control  simultaneously  )  by  the  user.  By  design,  each  utterance 
v/as  repeated  approximately  20-23  times  in  different  sets  within  a 
specific  grammar.  It  was  also  recognized  that  although  some  of 
the  utterances  v/ere  unique  to  a  specific  grammar,  commands  such 
as  MControl_K"  were  common  to  all  grammars,  and  therefore  trained 
more  than  the  rest.  This  phase  was  the  most  demanding  anc  repetitive 
in  nature,  and  required  approximately  5.5  hours  to  complete. 
[Ref.  19] 

At  the  conclusion  of  the  enrollment  and  training  phases 
each  individual  received  a  30  minute  practice  session  while 
interfacing  v/ith  an  NWISS  wargaming  sample  scenario.  This  was  a 
monitored  exercise  stressing  verbal  consistency,  volume  control 
and  refamiliarization  v/ith  NWISS  commands.  For  a  complete  listing 
of  all  the  grammars,  the  individual  procedures  involved  with 
introducing  personnel  to  the  environment  and  all  of  the  individual 
words  that  were  enrolled,  refer  to  the  NWISS/VERBEX  3000  Users 
Guide,  Appendix  F. 

49 


4 .   Macintosh  Microcomputer  Training 

While  command  input  accuracy  in  the  vcice  environment  is 
based  on  repetition  coupled  with  consistency,  the  point  select 
method  inherent  in  Macintosh's  mouse  requires  manual  dexterity 
and  practiced  digit  manipulation.  To  allow  the  Macintosh  user  to 
interface  with  the  wargaming  scenario  resident  on  the  VAX  computer, 
a  program  was  written  in  the  language  BASIC  to  provide  the  user 
with  command  words  from  a  limited  vocabulary.  The  program  simulates 
the  VT100  terminal  in  that  it  allows  information  communicatee  to 
the  VAX  to  be  displayed  on  the  screen  for  analysis.  It  permits 
the  sequential  selection  of  words  ana  limited  phrases  through  an 
appropriate  column  header  and  its  associated  "drop-down"  window; 
all  operated  and  selected  via  mouse  technology.  A  capability  also 
exists  for  the  user  to  enter  the  manual  mode  and  type  commands  to 
the  computer  using  the  attached  keyboard. 

Training  for  the  Macintosh  was  accomplished  in  an  "offline" 
mode  which  allowed  the  user  to  make  full  use  of  all  operational 
comm.ands,  without  the  response  of  the  VAX  computer.  Each  user  was 
given  a  15  minute  orientation  brief  on  the  use  of  the  mouse,  and 
the  subsequent  self -generated  training  in  command  selection 
averaged  35-50  minutes. 

H.   PRACTICE  SESSIONS 

1.   Initial  Individual  Tasks 

After  training  their  individual  speech  patterns  the 
subjects  were  given  written  instructions  on  how  to  bring  up  the 
NWISS   game   and   initialize   the   VEPBEX   3000   and   Macintosh 
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microcomputer.  They  were  given  a  list  of  approximately  50  separate 
NWISS  tasks  to  perform  by  issuing  commands  using  voice  and  the  mouse 
peripheral  input.  While  the  lists  v/ere  different  for  the  continuous 
and  Macintosh  inputs  because  force  names  ana  call  signs  were 
different,  the  general  set  of  tasks  were  the  same.  The  tasks  v/ere 
very  specific  in  nature  to  require  the  subjects  to  put  them  into 
acceptable  NWISS  format,  anc  then  enter  the  commands  as  a  recog- 
nizable speech  or  Macintosh  input. 

There  v/ere  several  purposes  for  the  indivicual  practice 
sessions  described  above;  especially  for  the  VERBEX  3000  users. 
It  helped  to  identify  problems  with  recognition  (so  that  subjects 
could  retrain  vocabulary/refresh  their  memories),  and  it  was 
useful  in  teaching  subjects  to  take  a  general  task  and  convert  it 
into  NWISS  format.  This  was  especially  important  for  VERBEX  3000 
users  in  that  the  necessary  pauses  between  phrases  v/ere  required 
for  the  different  grammars.  Most  users  initially  thought  they 
were  having  recognition  problems,  when  in  fact  they  were  not 
pausing  in  the  proper  places.  The  authors  participated  in  these 
practice  sessions  (one  on  the  VERBEX  3000  and  one  on  the  Macintosh) 
so  that  a  better  feel  for  the  problems  encountered  by  the  subjects 
could  be  experienced  first  hand.  [Ref.  2] 

Enhancement  of  the  experimental  design  v/as  achieveo  by  using 
subject  training  that  allowed  the  identification  of  "obscure" 
NWISS  vocabulary  not  often  trained  by  the  VERBEX  300C  users;  or 
inadvertently  overlooked  by  the  Macintosh  subjects.  There  was  an 
additional  problem  with  VERBEX  3000  because  while  an  individual 
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word  may  have  been  trained,  it  may  not  have  been  included  ir.  all 
grammars  or  phrases.  These  imposed  limitations  were  taken  into 
account  at  the  scenario  design  phase  by  Wright  and  Hanson  ir; 
their  thesis,  and  since  the  same  scenarios  were  used  ir.  this 
thesis  they  were  also  fully  incorporated. 
2 .   IHilSS  Practice  Sessions 

All  subjects  went  through  a  practice  NWISS  session  before 
the  actual  experimental  testing,  and  (as  described  in  Chapter  VI) 
all  subjects  were  assigned  to  either  continuous  voice,  Macintosh 
microcomputer  or  keyboard  entry.  There  v/ere  several  reasons  for 
this  practice  session.  Since  all  of  subjects  had  recently  completed 
a  separate  classroom  experiment  using  NWISS.  it  was  necessary  to 
ref ami liarize  the  voice  users  with  the  names  of  the  forces  for 
which  they  had  previously  trained  their  voice  patterns.  In  addition, 
it  was  found  that  in  the  majority  of  cases  (cue  to  the  large 
number  of  students  present  in  each  group),  all  the  subjects  were 
not  able  to  rotate  through  the  specific  position  that  allowed 
them  to  actually  input  commands  during  the  running  of  the  session. 
This  sample  scenario  gave  each  individual  an  inkling  of  the  type 
of  commands  to  be  performed  during  the  actual  experiment.  All 
subjects  had  access  to  status  boards  showing  their  available 
forces,  which  made  the  remembering  of  names  less  of  problem  with 
continued  practice. 

The  practice  sessions  proved  to  be  very  instructive  for  the 
authors  during  the  actual  preparation  for  the  experiment.  These 
sample  sessions  used  the  actual  test  experiment  format,  and  the 
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observance  of  all  the  groups  in  a  "dry  run"  insured  that  procedures 
used  later  would  be  acceptable.  It  also  served  to  verify  that 
there  was  ample  time  to  complete  the  session  and  gather  resultant 
data  in  the  scheduled  50  minute  periods.  Ml  environmental  conditions 
were  examined,  and  the  correct  positioning  of  stereo  speakers  ano 
settings  on  rheostats  was  noted  to  insure  equal  noise  and  light 
at  both  positions.  This  also  served  to  insure  that  the  settings 
could  be  easily  duplicated  during  the  actual  sessions. 
3.   Conduct  of  the  Practice  Session 

The  authors  had  prepared  the  laboratory  before  the  subjects 
arrival  so  that  all  equipment  was  turned  on,  the  appropriate 
NWISS  scenario  v/as  initialized  and  placed  in  pause  mode  and  the 
voice  patterns  for  the  intended  VERBEX  3000  user  v/ere  loaded  into 
the  system.  A  command  to  save  the  session  was  included  for  post 
analysis  to  insure  that  the  desired  data  v/as  actually  being 
collected. 

Before  the  start  of  the  practice  session,  the  subjects  v/ere 
given  an  administrative  briefing  about  the  practice  run,  and 
concerning  specific  instructions  for  the  test  sessions.  The  exact 
items  covered  are  shov/n  in  Appendix  C.  Following  the  administrative 
briefing,  the  subjects  were  briefed  on  the  particular  practice 
scenario.  Each  individual  v/as  supplied  with  a  sample  list  of 
NWISS  commands  to  be  used  for  reference  during  the  session,  if 
needed.  A  verbal  briefing  on  the  mission  v/as  also  given  to  all 
subjects,  and  any  questions  were  answered. 
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The  KWISS  game  was  allowed  to  run  for  approximately  30 
minutes.  At  the  completion,  the  subjects  departed  and,  if  another 
croup  was  to  come  in,  the  authors  prepared  the  laboratory  accord- 
ingly. Additionally,  copies  of  the  photographed  came  screens 
were  printed  at  this  time.  As  was  previously  mentioned  this 
procedure  was  only  performed  for  the  first  fev;  sessions  to  verify 
correct  information,  ano.  to  determine  how  long  the  entire  analysis 
process  would  take.  Printouts  were  not  necessary  for  all  the 
practice  sessions  because  the  data  would  not  be  used  in  the  experiment. 
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VI.       CONDUCT   OF    THE    EXPERIMENT 

A.  INTRODUCTION 

This  chapter  will  Gescribe  the  layout  and  control  of  the 
laboratory,  the  actual  conduct  of  the  experiment  and  the  method 
used  to  record  the  individual  sessions. 

B.  LABORATORY  ENVIRONMENT 

1.   Configuration 

The  NWISS  Command  and  Control  laboratory  was  divided  into 
two  independent  bays  (each  configured  approximately  sixteen 
feet  high  by  eighteen  feet  in  length)  and  was  used  to  simulate 
the  environment  for  the  experiment.  The  following  equipment  for 
each  of  the  two  input  modes  was  selectively  located  in  its  respective 
section: 

1)  a  Ramtek  graphics  screen  with  a  separate  stereo  speaker 
beside  it  for  producing  background  noise, 

2)  a  VT100  game  status  terminal  which  was  used  to  retrieve 
game  information  only,  and 

3)  a  player  terminal  (VT102  for  the  Macintosh  or  ADM-31  for 
the  voice)  which  allowed  commands  to  be  physically  entered 
into  the  computer. 

Using  the  ADM-31  as  a  validation  device  for  character  strings 

enroute  to  the  computer,  the  voice  subjects  v/ere  able  to  monitor 

their  inputs,  change/cancel  incorrect  entries  and  use  the  keyboard 

(on  isolated  occasions)  in  the  event  vcice  recognition  proved  to 

be  an  insurmountable  problem.  The  laboratory's  physical  layout 

provided  the  subjects  with  easy  access  to  all  terminals,  and  the 

experiment  proctors  could  evaluate  the  progress  of  each  session 
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using  the  VAX/NWISS  control  station  in  the  Control  Eay.  Additional 
equipment  was  located  in  each  bay,  but  was  separatee  by  a  partition 
u.id  net  directly  associated  with  the  experiment  itself  (see 
Eigure  6.1  for  a  complete  Laboratory  Layout). 
2.   Laboratory  Control 

Personnel  traffic  v/as  not  restricted  during  the  individual 
vc ice/Hac intcsh  practice  sessions,  but  during  the  actual  test 
sessions,  access  to  the  laboratory  was  more  restrictive  in  nature. 
While  the  experiment  was  in  progress  all  personnel  v/ere  requested, 
to  use  the  rear  entrance  to  preclude  the  distraction  of  the 
pneumatic  sliding  door,  and  to  prevent  undesirable/uncontrolled 
light  from  entering  the  controlled  surroundings.  Laboratory  users 
not  involved  in  the  experiment  were  advised  that  use  of  the 
facility  v/as  defaulted  to  whatever  noise  and  light  environment 
was  currently  in  use  during  the  experiment.  Due  primarily  to  this 
precautionary  measure,  the  environmental  conditions  v/hich  had 
been  adjusted  and  measured  before  each  run  v/ere  adequately 
preserved. 

C.   TEST  SESSIONS 

The  procedure  for  the  actual  test  sessions  was  an  exact  dupli- 
cation of  the  practice  sessions.  As  before,  prior  to  the  arrival 
of  the  subjects,  the  proper  NWISS  scenario  was  initialized  and 
the  vcice  patterns  for  the  subjects  scheduled  v/ere  loaded  into 
the  machines  (see  Appendix  E) .  The  subjects  were  briefed  in 
accordance  with  Appendix  C  and  provided  with  a  copy  of  the  applicable 
scenario  for  their  perusal.  Any  peculiarities  within  a  given 
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Figure  6.1    Laboratory    Configuration 


57 


scenario  v/hich  the  subjects  v/oulc  not  normally  have  experienced 
previously  v/ere  pointeo  out,  and  any  questions  were  answerec. 

Following  the  briefing,  the  actual  NWISS  session  began.  After 
the  subjects  went  to  their  positions,  the  proper  noise  and  lighting 
levels  were  set,  and  the  game  was  started.  All  the  sessions  v/ere 
run  for  forty  minutes.  Personnel  who  had  repeated  difficulties 
v/ere  given  assistance  as  necessary.  At  the  conclusion  of  each 
run,  copies  of  the  game  sessions  were  printed  out  ano  annotated 
for  later  classification.  The  forty  minute  game  time  point  was 
noted  on  the  printouts,  and  the  number  of  commands  and  "Control_K ' s" 
entered  v/ere  computed.  [Ref.  2] 

While  the  majority  of  the  experiment  sessions  were  completed 
as  planned,  some  rescheduling  was  needed  due  to  the  absence  of 
personnel,  or  to  some  unexpected  NWISS  game  problems.  Rescheduling 
of  personnel  created  no  particular  problems  because  each  input 
method  was  used  to  play  an  individual  NWISS  session,  and  the 
proper  scenario  and  environmental  conditions  could  be  reset  for 
any  individual  makeup  as  required. 
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VII.    DATA  ANALYSIS 

A.  INTRODUCTION 

The  purpose  of  this  experiment  was  to  evaluate  the  effectiveness 
of  two  computer  input  devices  uncer  various  environmental  conoitions. 
Since  a  wargame  scenario  was  used,  the  conditions  sinulated 
tactical  or  Command  Information  Center  (CIC)  environments.  The 
hypothesis  for  the  experiment  was  there  woulc  be  no  significant 
difference  between  using  the  VERBEX  3000,  the  Macintosh  or  the 
keyboard  systems.  As  stated  earlier,  the  results  were  based  on 
the  positive  Measure  of  Effectiveness  of  comparing  the  total 
number  of  commands  entered,  and  the  total  number  of  mistakes  made 
while  entering  the  commands. 

B.  DATA  SUMMARY 

The  number  of  commands  entered  was  determined  by  summing  the 
"order  entered"  lines.  As  previously  noted,  the  only  exception 
was  the  aircraft  launch  sequence  which  v/as  counted  as  either  four 
or  five;  depending  on  v/hether  or  not  "weapons  v/ere  loaceo".  The 
mistakes  were  determined  by  summ.ing  the  applicable  Control_K's 
that  v/ere  correctly  and  deliberately  entered.  Raw  data  for  the 
experiment  is  listed  in  Appendix  E. 

Table  7.1  is  the  key  which  describes  the  environmental  conditions 
used  in  the  experiment.  It  should  be  used  as  a  reference  when 
analyzing  the  other  accompanying  tables  and  figures  in  this  chapter. 


59 


TABLE  7.1 

ENVIRONMENTAL  CGND IT IONS 

NOISE: 

LOW   -   65  Decibels  (cb) 
HIGH   -   8  5  Decibels 
LIGHT: 

NORMAL  -  1  Foot-lambert 
LOW    -  .05  Foot-lambert 


Table  7.2  summarizes  the  mean  number  of  commands  entered, 
based  on  the  variable  noise/light  conditions.  Figure  7.1  oisplays 
the  data  in  graphic  form. 

TABLE   7 . 2 

MEAN  NUMBER  OF  COMMANDS  ENTERED 
Environmental  Condition:  Noise/Light 
Input       Low    Std    Low    Std    High    Stci    High   Stc 
Device     Norm  Dev   Low   Dev   Norm   Dev   Low   Dev   Avq 


Verbex      55.3   23.4   69.8   23.4   40.8    15.6   60.7   27.3   56.7 


Macintosh   28.2    9.4   43.4   10.0   44.4    16.8   47.2   11.8   40.8 


Keyboard    74.1   15.9   71.7   19.6   79.0    27.8   71.5   19.5   74.1 
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Table  7.3  summarizes  the  i'iean  Number  of  Control_K's  entered, 
based  or.  the  variable  noise/light  conditions.  Figure  7.2  displays 
the  data  in  craphic  form. 


TABLE   7 . 3 

MEAN  NUMBER  OF  COKTROL_K ' S  ENTERED 
Environmental  Condition:  Noise/Light 
Input       Low   Std   Low   Std   High   Std    High   Std 
Device      Norm  Dev   Low   Dev   Norm   Dev   Low   Dev   Avc 


Verbex       8.3    8.7    4.5    2.3    5.3     3.8    5.3    4.1  5.9 

Macintosh    7.8    3.7    5.8    2.4    6.2     3.5    7.8    4.2  6.9 

Keyboard     5.6    4.3    7.7    8.0    7.6     3.6    7.3    6.4  7.0 

Table  7.4  summarizes  the  Mean  Number  of  Con.mands  entered  based 

on  the  trial  number.  Figure  7.3  displays  the  data  in  graphic  form. 
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TABLE   7.4 

MEAN  NUMBER  OF  COMMANDS  ENTERED 
TRIAL  NUMBERS 
Input  Std  Std  Std  Std 

Device      1rst     Cev     2nd     Dev     3rd     Dev     4th    Dev 


Vertex  60.6  21.1  51.5  19.4  60.1  29.2  55.3  28.4 
Macintosh  27.9  9.2  38.6  15.4  47.0  7.6  51.3  12.2 
Keyboard    63.7     23.1    76.2    20.3    73.2    16.2    83.2   20.1 


Table  7.5  summarizes  the  Mean  Number  of  Control_K's  entered 
based  on  the  Trial  Number.  Figure  7.4  displays  the  data  in  graphic 
form. 

TABLE   7.5 
MEAN  NUMBER  CF  CONTROL  K'S  ENTERED 
TRIAL  NUMBERS 
Input  Std  Std  Stci  Std 

Device      1rst     Dev     2nd     Dev     3rd     Dev     4th    Dev 


Verbex 

8.7 

8.4 

6.8 

4.  1 

3.6 

2.3 

4.5 

3.4 

Macintosh 

7.2 

3.7 

7.2 

3.6 

6.9 

3.7 

6.4 

3.4 

Keyboard 

8.1 

5.5 

9.0 

8.3 

6.1 

4.3 

5.2 

3.4 
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C.  STATISTICAL  METHODS 

Once  the  data  from  the  experiment  was  compiled,  two-way  analysis 
of  variance  techniques  helped  evaluate  the  results.  Two  statistical 
analysis  software  programs  resident  in  the  IBM  3033  mainframe 
computer  were  usee  to  facilitate  the  calculations.  The  first  was 
MINITAB,  and  the  second  was  the  Statistical  Analysis  System 
(SAS)  . 

In  testing  analysis  of  variance  the  following  two  assumptions 
must  be  made:  1)  the  populations  from  which  the  samples  came  can 
be  approximated  clcsely  with  normal  distributions,  and  2)  they 
all  have  the  same  variance.  [Pef.  2C]  For  each  case,  the  hypothesis 
was  tested  at  a  confidence  level  of  95  percent.  Additionally,  for 
each  environmental  condition  (i.e.,  ncise/light  level),  a  ratio  of 
the  number  of  errors  that  can  be  expected  per  the  number  of  valic 
commands  entered  was  calculated.  This  was  computed  for  each  of 
the  three  input  devices.  Finally,  an  analysis  using  SAS  was 
conducted  as  a  backup  for  some  of  the  data  that  was  analyzed 
using  the  MIMTAB  program.  [Ref.  21] 

D.  ANALYSIS 

1.   Introouction 

As  with  the  statistical  analysis,  it  is  desirable  to  use 
as  many  data  samples  as  possible.  Since  it  was  difficult  to 
comjr.it  a  large  number  of  subjects  to  the  experiment,  each  of  the 
24  subjects  participated  in  four  separate  sessions,  thereby 
totaling  96  samples  (VERBEX  3000  and  Macintosh).  Again,  the 
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keyboard  data  v/as  that  which  was  compiled  from  the  Wright/Hanson 
experiment. 

It  was  now  important  to  ensure  that  the  scenarios  were 
variea  enough  to  be  an  "independent  session"  for  the  subjects,  so 
that  no  unfair  advantage  would  occur  during  a  trial  run.  The 
two-way  analysis  of  variance  v/as  obtained  between  the  number  of 
commands  entered  ana  the  different  scenarios  used.  Also,  a  two-way 
analysis  was  conducted  between  the  number  of  Control_K's  entered 
and  the  different  scenarios  used.  The  result  was  the  same  for 
each,  which  indicated  that  neither  one  exhibited  any  significant 
difference.  It  is  therefore  reasonable  to  assume  that  the  experiment 
consisted  of  9  6  individual  sessions.  The  Wr ight/Manson  experiment 
consisted  of  13  keyboard  operators  v/ho  also  played  the  identical 
set  of  4  different  scenarios  (totalling  52)  .  The  combined  total 
is  then  148  sessions  (samples)  for  this  experiment. 
2.   Best  Environment  for  Each  Input  Method 

It  v/as  expected  that  each  input  device  v/ould  do  better 
under  a  specific  combination  of  environmental  conditions.  For  two 
of  three  devices  the  results  also  shov/ed  that  the  combination 
that  provided  the  greatest  number  of  valid  commands  to  be  entered, 
also  provided  the  fev/est  number  of  errors  (i.e.,  Control_K '  s )  . 
This  is  a  desired  situation  and  is  reasonably  intuitive.  What 
follows  is  a  brief  description  of  the  best  environment  found  for 
each  particular  input  method.  See  Figure  7.5  for  a  listing  of  the 
ratios  (errors  per  valid  commands)  for  each  device. 
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For  the  VERBEX  3C00  system  the  greatest  number  of  valid 
commands  were  entered  under  a  low  noise,  low  light  condition. 
Concurrently,  the  fewest  errors  v/ere  entered  under  these  exact 
conditions.  Based  on  the  F  ratios  calculated  after  performing  a 
two-way  analysis  of  variance  (at  the  S5  percent  level),  tnere  v/as 
NO  significant  difference  between  the  number  of  commands  entered 
ana  the  environmental  conditions.  The  same  result  was  realized 
(NO  significant  difference)  between  the  number  of  Controi_K's 
(errors  inputed)  and  the  environmental  conditions.  That  is, 
although  there  is  a  "lowest  ratio"  for  the  VERBEX  3000  system, 
there  is  NO  significant  difference  between  the  number  of  commands 
entered  or  errors  entered,  and  the  environmental  conditions  under 
which  it  was  used.  As  expected,  the  lowest  ratio  (errors  per 
valid  command  entries)  came  under  the  same  low  noise,  lev/  light 
condition.  (See  Appendix  E  for  actual  Statistical  Data.) 

For  the  keyboard  system,  the  highest  number  of  valia 
commands  were  entered  under  tv/o  conditions:  1)  lew  noise  and 
normal  lighting,  and  2)  low  noise  and  lew  lighting.  Here  again, 
the  fewest  number  of  errors  were  made  under  low  noise  and  normal 
lighting  conditions.  Like  the  VERBEX  3000,  the  F  ratio  calculated 
from  the  two-way  analysis  of  variance  revealed  NO  significant 
difference  between  the  number  of  commands  entered  (and  the  number 
of  Control_K's  entered)  and  the  environmental  condition.  Never- 
theless, the  lov/est  ratio  occurred  under  a  low  noise,  normal 
lighting  condition. 
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For  the  Macintosh,  the  highest  number  of  valid  commands 
were  entered  under  high  noise,  low  lighting  conditions.  The 
fewest  number  of  errors  were  entered  under  low  noise,  lew  light 
conditions.  As  with  the  previous  input  devices.  NO  significant 
difference  was  revealed  between  the  number  of  commands  entered 
(or  the  number  of  errors  entered)  and  the  environmental  condition 
used.  Here,  the  lowest  ratio  occurred  while  operating  under  low 
noise  and  low  lighting  conditions. 

3 .   Best  Input  Method  for  Each  Environment 

This  question  was  answered  using  a  two-way  analysis  of 
variance.  Each  environmental  condition  (i.e..  noise/light)  should 
reveal  a  desired  input  device  which  yields  the  largest  number  of 
commands,  and  also  the  fewest  number  of  errors.  As  in  the  previous 
question,  the  ratio  of  errors  to  valid  commands  also  revealed 
some  practical  insight  into  which  device  may  be  preferable  when 
using  a  particular  noise/light  condition.  The  two  hypotheses 
stated  that  the  number  of  commands  entered  (and  the  number  of 
Control_K's  committed)  would  be  equal  for  all  three  input  devices. 

Conclusive  results  (based  on  the  F  ratio  at  95  percent) 
show  there  IS  a  significant  difference  between  the  number  of 
commands  entered  and  the  input  devices.  In  all  four  noise/light 
conditions  the  keyboard  operators  entered  more  commands  than  all 
the  other  subjects.  This  was  not  the  case,  however,  when  a  measure- 
ment of  the  number  of  Control_K's  committed  was  taken.  Here,  the 
two-way  analysis  of  variance  revealed  no  significant  difference 
among  any  noise/light  condition  and  the  input  devices.  Even  on 
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examining  the  raw  data  it  was  shown  that  NO  clear  advantage 
existed  for  any  particular  device.  The  calculated  ratios  (see 
Figure  7.5)  displayed  the  following  advantages  under  each  environ- 
mental condition: 

BEST  INPUT  METHOD 

1.  Low  Noise  /  Normal  Light KEYEOARD 

2.  Low  Noise  /  Low  Light VERBEX 

3.  High  Noise  /  Normal  Light  KEYEOARD 

4.  High  Noise  /  Low  Light VERBEX 

4.   Best  Overall  Input  Method 

This  question  was  answered  using  a  one-way  analysis  of 
variance.  The  F  ratio  was  again  calculated  at  a  95  percent  confidence 
interval,  and  the  hypotheses  were  the  same  as  the  ones  describee 
in  the  previous  question. 

The  results  revealed  a  significant  difference  between  the 
mean  number  of  commands  entered  and  the  input  devices,  thereby 
rejecting  the  null  hypothesis.  The  raw  data  shewed  that  the 
keyboard  operators  entered  more  commands  than  the  operators  of 
the  other  devices.  The  VERBEX  3000  players  were  only  slightly 
lower,  v/hile  the  Macintosh  players  entered  an  average  of  43 
percent  fev/er  commands  per  session. 

As  far  as  the  number  of  errors  committed  was  concerned, 
the  one-way  analysis  of  variance  showed  NO  significant  difference; 
thereby  accepting  the  null  hypothesis  that  the  number  of  Control_K's 
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entered  by  all  three  input  devices  was  equal.  The  rav/  data  showed 
only  a  difference  of  2  or  3  errors  amongst  input  devices  per  each 
session.  The  VERBEX  3000  had  the  fewest  (an  average  of  5.8),  next 
was  the  Macintosh  (with  6.9),  and  finally  the  keyboard  (with  8.1). 
5.   Possible  Learning  Curves 

It  was  interesting  to  note  that  NO  significant  learning 
curve  took  place  between  the  subjects  when  considering  either 
the  nunber  of  commands  entered,  or  the  number  cf  Control_K's 
committed.  It  is  v/orth  mentioning,  however,  that  the  Macintosh 
subjects  increased  the  number  of  commands  entered  for  each  succeeding 
trial  (session) .  This  can  be  seen  graphically  depicted  in  Figure 
7.3.  The  reverse  v/as  not  the  same  for  the  number  of  Control_K's 
that  the  subjects  entered,  and  again,  neither  was  it  significant 
(this  can  be  seen  graphically  in  Figure  7.4). 

B.   SUMMARY 

To  summarize,  there  were  few  significant  results.  Fven  with 
the  adceo  benefit  of  looking  at  the  rav/  data,  not  all  the  questions 
were  answered  unequivocally.  None  of  the  three  input  Devices 
could  be  labeled  as  a  clear-cut  winner  over  any  of  the  other 
devices.  Each  of  the  devices  performed  best  given  a  specific  set 
of  environmental  conditions.  Oddly  enough,  the  VERBEX  3000  and 
Macintosh  both  performed  best  in  the  Low  Kcise/Low  Light  conditions, 
and  the  keyboard  performed  the  best  in  the  High  Noise/Normal 
Light  conditions.  Without  regard  to  the  Noise/Light  conditions, 
the  VERBEX  3000  system  had  the  lowest  ratio  (i.e.  ,  error  to  command)  , 
with  the  keyboard  a  close  second  (.102  to  .112  respectively).  The 
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Macintosh  entered  fewer  comr.ands  than  the  others,  and  was  ranked 
second  v/hen  measuring  the  nur.bers  of  errors  committed.  Additionally, 
no  significant  learning  took  place  curing  the  four  separate 
sessions  that  each  individual  participated  in  during  the  experiment. 
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VIII.   CONCLDSIONS  AND  RECOMMENDATIONS 

A.  INTRODUCTION 

The  objective  of  this  thesis  was  to  evaluate  different  input 
interfaces  to  the  NWISS  wargaming  scenarios  resident  on  trie  VAX 
11/780  computer.  The  two  hypotheses  were  that  the  mean  number  of 
commands  entered  and  the  mean  number  of  errors  committed  (using 
any  of  the  three  tested  input  devices)  woula  be  equivalent.  While 
the  objectives  of  the  thesis  v/ere  successfully  established  on  the 
completion  of  the  experiment,  no  single  input  device  can  be 
designated  as  the  "better"  of  the  three. 

B.  CONCLUSIONS 

Before  the  experiment,  the  authors  of  this  thesis  anticipated 
that  the  voice  system  would  outperform  the  other  devices.  By  the 
term  "outperform",  it  is  meant  that  the  VERBEX  3000  subjects 
would  enter  significantly  more  commands  and  commit  fewer  overall 
errors  than  the  other  devices.  It  was  expected  that  the  Macintosh 
would  outperform  the  keyboard  simply  by  virtue  of  its  current 
technology.  In  general,  the  analysis  of  the  data  did  not  confirm 
these  expected  outcomes. 

Many  reasons  can  be  given  to  explain  these  unexpected  results. 
As  far  as  the  keyboard  was  concerned,  most  of  the  subjects  wasted 
little  time  in  moving  their  keyboards  as  close  as  possible  to 
their  data  input  (CRT)  screen  in  the  low  light  condition.  Using 
the  video  screens  inherent  glcw  helped  considerably  in  locating 
the  correct  keys;  especially  for  the  non-typists.  Also,  a  post 
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experiment  survey  revealed  that  the  keyboard  operators  were 
experienced  typists  by  and  large.  By  this  it  is  meant  that  at 
worst  most  of  the  subjects  v/ere  able  to  use  both  hands  while  only 
occasionally  looking  at  the  keys.  By  implication,  military  officer 
postgraduate  students  are  collectively  more  likely  to  have  had 
typing  experience  (i.e.,  used  typing  skills  during  their  lengthy 
education/training)  than  the  general  population.  Their  average 
lifetime  exposure  to  keyboard  technology  is  high,  and  again  this 
particular  group  was  above  the  norm.  [Ref.  2]  On  the  other 
nana,  none  of  the  subjects  had  prior  experience  v/ith  either  the 
continuous  voice  system,  and  only  a  few  subjects  had  prior  experience 
with  the  Macintosh  microcomputer  system.  This  brings  into  focus 
the  implications  of  the  necessary  training  involved  for  each 
device.  The  VERBEX  3000  subjects  trained  with  the  voice  system  an 
average  of  six  hours  only  one  week  before  the  experiment.  Consioering 
that  it  takes  years  of  frequent  typing  to  become  proficient,  the 
VERBEX  3000  subjects  did  v/ell  to  keep  up  with  the  keyboard  operators. 
The  Macintosh  subjects  practiced  for  about  35  minutes  each, 
concentrating  totally  on  developing  the  motor  skills  necessary  to 
maneuver  the  arrow  using  the  mouse  peripheral.  In  retrospect,  it 
is  felt  that  they  would  have  performed  considerably  better  if 
they  had  been  as  proficient  (through  additional  training)  in  the 
use  of  the  mouse  as  the  keyboard  subjects  were  in  using  their 
acquired  typing  skills.  The  raw  data  from  the  Macintosh  players 
shov/ed  a  slight  increase  in  the  mean  number  of  commanc.s  entered 
for  each  individuals1  succeeding  trial.   Although  it  was  not  a 
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significant  increase,  the  same  statement  could  not  be  mace  fcr 
the  other  devices. 

When  looking  at  the  ratios  (see  Figure  7.5),  each  cevice 
performed  v/ell  under  certain  noise/light  conditions.  The  VERBEX 
3C00  and  the  Macintosh  had  the  lowest  error  tc  command  ratio 
under  the  lov;  noise/lev/  light  conditions.  Not  surprisingly,  the 
keyboard  operators  had  the  lowest  ratio  under  the  high  noise/normal 
light  conditions.  For  the  keyboarc  and  VERBEX  operators,  each  had 
the  lowest  error  to  command  ratio  in  two  of  the  four  environmental 
conditions.  Looking  specifically  at  the  "light  conditions"  (without 
regard  to  noise)  ,  the  VERBEX  3000  system  had  a  lower  ratio  than 
the  other  devices.  Similarly,  the  keyboard  users  attained  their 
lowest  ratio  under  "normal  light  conditions".  Considering  the 
inexperience  of  the  VERBEX  3000  operators,  they  did  extremely 
well  in  the  most  likely  high  stress,  wartime  environment  (high 
noise,  lov;  light).  The  error  to  command  ratio  v/as  .015  lev/er  than 
the  second  best,  keyboard. 

Although  we  can  logically  conclude  that  the  keyboard  die 
perform  better  than  the  other  devices,  it  would  be  illogical  to 
conclude  that  the  other  input  devices  do  not  warrant  further 
research.  Just  looking  specifically  at  noise  conaitions  (without 
regard  to  the  lights)  the  VERBEX  3000  system  had  a  lower  ratio 
than  the  other  devices.  Considering  the  inexperience  of  the 
VERBEX  3000  operators,  their  ratio  was  only  .004  higher  than  the 
keyboard  subjects  in  the  most  likely  high  stress  wartime  environment 
(that  of  high  noise  and  lov/  light  conditions)  . 
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Another  area  analyzed  was  the  presence  of  any  learning  trends 
the  subjects  may  have  received  from  previous  sessions,  [.'one  of 
the  subjects  performed  significantly  better  over  tiie  four  trials. 
While  this  was  not  necessarily  an  encouraging  trend,  this  result 
supports  the  independence  of  each  session,  thereby  making  the 
sample  size  of  the  experiment  itself  mere  valid. 

One  final  comment  should  be  made  about  the  likelihood  that 
inexperienced  operators  will  often  be  at  the  controls  of  the 
input  media  in  an  academic  or  training  environment.  For  example, 
a  senior  officer  (i.e..  0-5  or  above)  attending  the  Naval  War 
College  will  play  similar  war  games.  It  is  highly  unlikely  that 
an  individual  will  have  had  prior  experience  using  speech  or 
"menu-oriented"  systems  for  input  communication.  It  can  therefore 
be  concluded  that  with  a  minimum  of  introduction  to  these  devices, 
almost  equal  performance  can  be  attained  to  that  of  the  experienced 
typist  if  the  syntax  required  of  the  game  commands  is  also  well 
learned  by  the  novice  user. 

C.   RECOMMENDATIONS 

It  is  the  opinion  of  the  authors  that  further  study  is  justified 
concerning  the  potential  of  the  three  input  devices.  Before 
another  experiment  is  performed,  however,  it  is  recommended  that 
sufficient  training  of  the  subjects  be  conducted.  It  was  felt 
that  the  Macintosh  subjects  would  have  performed  significantly 
better  with  more  extensive  training  and  dedicated  practice  sessions. 
This  would  have  enabled  them  to  become  more  familiar  with  the 
coordination  required  to  effectively  maneuver  the  cursor  with  the 
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mouse.  The  problem  with  this  proposal  is  the  time  constraint 
imposed  on  both  the  planning  and  execution  of  the  experiment. 
Optimally,  all  subjects  should  be  equally  skilled  in  the  use  of 
the  particular  input  device. 

Additionally,  the  softv;are  interface  program  for  the  Macintosh 
should  be  modified  to  facilitate  a  quick,  efficient  and  accurate 
mode  of  input  for  the  user.  This  modification  v/oula  require  a 
more  natural  arrangement  of  commands  to  allov/  an  easier  interaction 
with  the  sequence  of  these  commands  required  by  the  syntax  design 
of  the  softv/are. 

Acquisition  costs  for  these  devices  and  any  study  of  cost/benefit 
relationships  v/ould  provide  additional  fertile  ground  for  future 
research.  Current  downward  trends  in  hardware  costs,  and  a  general 
lowering  of  costs  for  all  technologies  which  are  driven  by  micro- 
computer advances  in  the  marketplace,  cause  a  "concept  of  costs" 
for  these  systems  to  be  well  worth  consideration. 

NWISS  should  be  used  as  a  medium  for  any  further  experimentation. 
The  necessary  peripherals  are  all  established,  and  it  provides  an 
accurate  simulation  of  a  real-time  environment.  Varying  environ- 
mental conditions  are  easily  simulated  in  the  C3  laboratory,  and 
the  accessibility  limitation  makes  it  an  ideal  place  to  conduct 
further  experiments. 

In  conclusion,  future  experiments  should  include  the  previously 
suggested  modification  to  the  Macintosh  software  and  the  most 
advanced  "touch  screen"  and  voice  technology  available.  It  is 
felt  that  these  three  devices  have  the  greatest  potential  to  meet 
the  requirements  for  an  efficient  input  medium. 
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D.   SUMMARY 

Han's  zeal  to  accomplish  the  ideal  "man-machine"  interaction 
has  allowed  significant  progress  to  cccur  regarding  input  devices. 
The  keyboard  was  first,  and  has  been  dependable  but  restrictive; 
sometimes  tc  the  point  of  being  inconvenient.  Ouick  to  approach 
popularity  were  several  "non-traditicnal"  methods;  one  being  the 
voice  system.  The  discrete  voice  system  is  unnatural  and  hard  to 
use  for  most  users,  while  the  continuous  system  is  easier  to  use 
and  more  convenient  for  the  jobs  requiring  user  input  and  manual 
labor  simultaneously.  Two  other  methods  are  the  "pull-down"  menu 
and  the  "touch  screen"  technologies.  Both  have  been  successful 
even  in  their  early  stages  of  development. 

Man  is  quickly  approaching  the  doorstep  of  Fifth  Generation 
computer  technology.  Therein  lie  "intelligent  systems"  which  will 
give  increasing  flexibility  to  input  devices  of  the  future. 
Unfortunately,  there  can  be  no  "perfect"  universal  input  method 
due  primarily  to  the  varied  requirements  and  restrictions  innate 
to  each  individual  job.  Nevertheless,  it  follows  that  continued 
research  should  be  aimed  at  producing  a  body  of  knowledge  v/hich 
would  allow  us  to  pick  the  best  method  of  computer  interaction  in 
a  specific  environment.  Although  it  is  recognized  that  computer 
speeds  today  are  restricted  from  full  use  because  of  the  slowness 
of  interface  hardv/are,  the  rate  of  progress  in  developing  high 
speed  CPUs  is  compounding  the  problem.  Therein  lies  the  impetus 
to  continue  toward  the  development  of  better  input  media. 
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APPENDIX  A 
SCENARIO  BRIEFINGS 

SCENARIO  A 


SITUATION:  Hostilities  have  alreacy  been  declared,  and  Orange 
forces  are  providing  escort  for  a  large  convoy  of  cargo  ships 
carrying  nuclear/chemical  weapons. 

FRIENDLY  FORCES:  You  have  available  one  Aircraft  Carrier,  2  Knox 
class  destroyers,  a  Los  Angeles  class  submarine  (positioneo  north 
of  the  convoy)  and  2  flights  of  4  P3C  aircraft. 

ENEMY  FORCES:  Orange  forces  have  8  ships  escorting  10  merchant 
ships  in  the  convoy.  No  air  assets  are  presently  available,  or 
expected, 

FISSION:  Destroy  all  Orange  combatants  without  inflicting  dan. age 
to  any  cargo  vessels  (due  to  the  nature  of  their  cargo) . 


HOE:      POSITIVE 

—  number  of  weapons  fired  during  the  session 

—  number  of  combatants  identified,  hit  and  damaged/sunk 

NEGATIVE 

—  number  of  merchant  ships  hit/sunk 

—  number  of  Elue  Aircraft  lost 

—  number  of  Elue  vessels  hit/sunk 
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SCENARIO  B 


SITUATION:  Approximately  20  Orange  merchant  vessels  are  anchored 
in  port  v/aiting  to  unload  various  cargo.  Hostilities  are  imminent. 

FRIENDLY  FORCES:  Blue  forces  include  2  P3C  aircraft  and  two 
groups  of  5  A6E's.  An  aircraft  carrier  is  available  to  supply 
further  air  assets  if  deemed  necessary. 

ENEMY  FORCES:  Location  and  makeup  of  Orange  forces  is  undetermined 
due  to  poor  weather/intelligence. 

MISSION:  Destroy  as  many  merchant  ships  as  possible,  anc  attack 
combatants  only  if  fired  upon. 


KOE:     POSITIVE 

—  number  of  weapons  fired 

—  number  of  merchant  ships  identified,  hit  and  damaged 

—  number  of  combatants  identified,  hit  and  damaged 

NEGATIVE 

—  number  of  Blue  forces  damaged/lost 

—  number  of  merchant  ships  remaining 
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SCENARIO  C 


SITUATION:  There  are  confirmed  reports  that  large  nur.bers  of 
Orange  Combatant  units  have  left  port,  are  running  EMCON  silent 
and  are  unlocatable  at  this  tine.  There  are  friendly/neutral 
forces  in  the  neighboring  area  ana  tension  is  rising.  No  hostilities 
have  been  declared  as  yet. 

FRIENDLY  FORCES:  1  F3C  aircraft  and  4  F14A  aircraft  are  available. 
Other  assets  are  in  the  area,  but  are  not  uncer  your  direct 
control.  Any  merchant  ships  encountered  are  not  to  be  consioered 
hostile. 

MISSION:  Search  for  possible  contacts  in  the  area  from  35N  to 
42Nf  and  from  174E  to  180£  (or  164E  to  170E  depending  on  the 
oceanic  displacement).  Locate,  identify  and  designate  as  many 
vessels  as  possible  as  either  enemy,  friencly  or  neutral.  Center 
the  plot  upon  the  P3C  at  the  start  of  the  session,  and  v/eapcns 
will  remain  tight  unless  fired  upon. 


MOE:     POSITIVE 

—  number  of  Orange  forces  properly  identified 

—  number  of  Elue  forces  properly  identified 

—  number  of  Neutral  forces  properly  identified 

NEGATIVE 

—  number  of  forces  improperly  designated 

—  number  of  forces  remaining  not  found/properly  designated 
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SCENARIO  D 


SITUATION:  An  Orange  SAG  is  100  MM  directly  north  of  your  position 
and  you  are  tasked  to  intercept  and  visually  identify  it.  Hostilities 
have  not  been  ceclared  as  yet,  but  are  expected  to  break,  out  scon. 

FRIENDLY  FORCES:  One  Aircraft  Carrier  (with  its  associatec.  aircraft)  , 
3  escort  ships  ano  a  submarine. 

ENEMY  FORCES:  The  strength  of  the  Orange  forces  is  unknown,  but 
they  are  not  expected  to  have  any  aircraft  launched  at  this  time. 

MISSION:  Launch  carrier  based  assets  to  search/identify  the 
suspected  Orange  SAG.  All  weapons  will  remain  tight  unless  fired 
upon. 


MOE:      POSITIVE 

—  number  of  aircraft  successfully  launched 

—  number  of  enemy  assets  properly  identified 

—  number  of  enemy  assets  damaged  (only  if  fired  upon) 

NEGATIVE 

—  number  of  Orange  forces  not  designated  as  enemy 

—  number  of  Elue  ships  hit/lest 

—  number  of  Blue  aircraft  lost 
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APPENDIX  B 
EXPERIMENT  TEAM  MEMBERSHIP/SCHEDULE 


Team 

1 

2 

3 

4 

5 

6 

7 

5 

9 
10 
11 
12 


Verbex 

F  i  n  ci  1  a  y 

G  a  h  a  g  a  r. 

Prautzch 

Orcutt 

Goyclen 

H  e  r  k  e  r  t 

Kotov/ski 

Cogliandro 

Thompson 

Euettner 

Hill 

Ocain 


Macintosh 

Caldwell 

R  o  b  e  r  t  s 

Stone 

W  i  c  k  s 

Alecxih 

M creau 

Kennedy 

McLaughlin 

Tekel 

Schuh 

Wall 

Gruetzner 


EXPERIMENT  SCHEDULE 


Time  10700  1  0800  |  0900  f  1000  j  1  100  j|  1200 
Dat>^       I 


1A1 


1200   1400  ? 1500 

I      I 

2A4 


2B1 


1D3 


16  OCT  !  3B1    7D2 


7A3 


5D1 


17  OC 


5B3 


3D4 


J. 


1 

|  5A4 


3C3 


1B2 


18  OCT 


6A1 


2D2  j  6D3 


5C2 


21  OCT    8D2  j  2C3 


22  OCT 


4D1  I  4C2 


CJ 


7C1  I  8C1  I  734 


9D1 


4A4 


3  OCT 


24  OCT 


1  SA3 

4 i h 


8B4 


6B2    9C2  {    6D4 


11B1 


9B4 


j  11C4   1CC1  I  1CD3 


- 


4. 


25  OCT 


28  OCT  I 


11Ai 


1  1D3 


j  10A2  f  12A1  ^1233  |12C< 


12D2  j 


1 

?  10B4 
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APPENDIX  C 
SUBJECT  ADMINISTRATIVE  BRIEFING 


NOTE:  The  follov/ing  administrative  briefing  was  initially  given 
to  each  subject  prior  tc  the  beginning  of  the  experiment.  Key 
points  were  re-emphasized  at  each  session  as  applicable. 

1.  The  sessicns  would  be  approximately  50  minutes  in  length,  each 
subject  would  be  scheduled  for  four  individual  sessions  and  a 
master  schedule  of  all  sessicns  was  posted  on  CDR.  Stewart's 
office  door  for  reference,  and  to  facilitate  changes  throughout 
the  experiment. 

2.  Subjects  were  asked  to  wait  outside  the  laboratory  until  told 
to  enter.  This  precaution  was  taken  in  order  to  preclude  an 
inadvertent  disturbance  in  the  environment  if  a  previous  group 
session  was  still  in  progress. 

3.  The  voice  patterns  for  the  scheduled  VEREEX  3000  subject  were 
already  loaded  to  avoid  any  unnecessary  delay.  The  user  simply 
had  to  set  the  gain  in  order  to  be  ready  to  start  the  session. 

4.  The  noise/light  conditions  for  the  particular  session  were 
explained  and  set. 

5.  The  subject  was  provided  with  the  scenario  for  the  session 
and  all  key  points  were  verbally  emphasized. 

6.  All  subjects  were  advised  to  ask  for  help  during  the  session 
if  any  problems  were  encountered. 

7.  The  subjects  were  asked  not  to  discuss  any  of  the  scenarios 
following  their  sessions  until  the  experiment  was  completed. 

8.  Each  group  was  reminded  of  their  next  scheduled  laboratory 
period  following  the  completion  of  a  session. 
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APPENDIX  D 

EXAMPLES  OF  COMMAND  SEQUENCES 

"Display  ...  Radius  2  5  0" 
"Execute  BAIRASW" 
"Execute  F14STRCAP" 

"For  Kitty  ...  Launch  ...  2  A7E  A6E  ...(course)  2  3  5... (speed) 

4  3  0  ...  (altitude)  15  0  0" 

Fit.  Plan:  "Mission  Strike" 

Fit.  Plan:  "Proceed  position  ...  4  5  N  ...  1  1  5  -  2  0  E" 

Fit.  Plan:  "Load  ...  (equipment)  3  Shrik  ...  6  MK82" 

Fit.  Plan:  "Stop" 

"Designate  Enemy  ...  (track)  E  E  0  0  5" 

"Display  ...  Place  Circle  Force  ...  M  P  6  1  5   (radius)  5  0" 

"For  V  A  0  1  0  ...Weapons  Free  Enemy  All" 

"For    S    H    2    0    0    ...    Earrier    ...(position)    15-20   N   ...    125 

-   1    5    E    ...     (bearing)    1    3    5    ...     (distance)    4    5    ...     (usinc) 

5  S   Q   6    2    . . .     (spacing)    4" 


NOTE:  Words  in  (parenthesis)  are  prompts  that  appear  en  the 
speech  unit. 
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APPENDIX  E 
INDIVIDUAL  DATA 


Environmental  Concition 

1  =  Low  Noise,  normal  Light;  2  =  Lev;  Noise .  Low  Li  grit 

3  =  High  Noise.  Normal  Light;  4  =  High  Noise.  Low  Light 


Input 

Env 

Team 

Device 

Scenario 

Condition 

C  n  a  s 

Control  K 

Trial  it 

1 

Vertex 

A 

1 

80 

6 

1 

B 

2 

78 

7 

2 

C 

4 

63 

1  1 

3 

D 

3 

98 

8 

4 

Macintosh 

A 

1 

27 

7 

1 

E 

2 

56 

1  1 

2 

C 

4 

26 

4 

3 

D 

3 

38 

9 

4 

2 

Verbex 

A 

4 

85 

16 

1 

B 

1 

78 

34 

2 

C 

2 

75 

3 

D 

3 

27 

1 

4 

Macintosh 

A 

2 

52 

8 

3 

B 

4 

56 

c 

4 

C 

3 

35 

6 

2 

D 

1 

38 

1  1 

1 

3 

Verbex 

A 

2 

64 

9 

1 

B 

1 

67 

10 

2 

C 

3 

36 

12 

3 

D 

4 

36 

5 

u 

Macintosh 

A 

2 

37 

5 

1 

B 

1 

28 

13 

2 

C 

3 

56 

1  1 

3 

D 

4 

49 

13 

4 

4 

Verbex 

A 

4 

42 

7 

4 

3 

3 

4  4 

1 

3 

C 

2 

62 

3 

2 

D 

1 

51 

4 

1 

Macintosh 

A 

4 

71 

7 

4 

B 

3 

54 

3 

C 

2 

37 

7 

2 

D 

1 

24 

5 

1 
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APPF. 

NDIX  E  (con 

t.  ) 

Input 

Env 

Tear:! 

Device 

Scenario 

Condition 

Cr,:tis 

Control  K 

Trial  /; 

5 

V  erbe;; 

A 

4 

7 

3 

— * 

■5 

E 

3 

18 

2 

1 

C 

2 

73 

5 

4 

D 

1 

39 

4 

2 

Macintosh 

A 

4 

37 

6 

3 

B 

3 

22 

14 

1 

C 

2 

29 

6 

4 

D 

1 

17 

8 

2 

6 

Verbex 

A 

1 

33 

8 

1 

B 

2 

112 

6 

3 

C 

4 

60 

1 

2 

D 

3 

45 

3 

4 

Macintosh 

A 

1 

27 

10 

1 

E 

43 

6 

3 

C 

4 

5C 

5 

2 

D 

-i 

42 

9 

4 

7 

Verbex 

A 

3 

19 

10 

1 

B 

4 

67 

1 

4 

C 

2 

81 

4 

3 

D 

1 

95 

6 

2 

Macintosh 

A 

3 

33 

4 

1 

B 

4 

45 

4 

4 

C 

2 

39 

2 

3 

D 

1 

35 

6 

2 

8 

Verbex 

A 

3 

40 

4 

B 

2 

27 

6 

3 

C 

4 

47 

5 

2 

D 

1 

37 

12 

1 

Macintosh 

A 

3 

39 

5 

4 

B 

2 

44 

6 

3 

C 

4 

32 

12 

2 

D 

1 

8 

2 

1 

o 

Verbex 

A 

3 

48 

4 

2 

B 

1 

48 

2 

4 

C 

2 

49 

3 

3 

D 

4 

50 

3 

1 

Macintosh 

A 

3 

44 

4 

2 

B 

1 

49 

2 

4 

C 

2 

38 

3 

3 

D 

4 

27 

5 

1 
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APPENDIX    E    (com.  ) 


Input 

Env 

Team 

Device 

Scenario 

Cot:  c;  it  ion 
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C  o  ti  t  r  o  1  K 

Trial  ft 

10 

V  e  r  b  e  x 

A 

2 

4  5 

0 

o 

s 

4 

81 

4 

4 

C 

1 

41 

5 

1 

D 

3 

63 

6 

2 
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A 

2 

46 

7 

■3 

B 

4 

61 

14 

4 

C 

1 

27 

10 

1 

D 

3 

29 

6 

2 

11 

V  e  r  b  e  x 

A 

2 

69 

4 

4 

B 

1 

17 

1 

1 

C 

4 

57 

6 

2 

D 

3 

39 
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3 
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2 

35 

5 
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B 

1 

29 

6 

1 

C 

4 

40 

3 

D 

3 

49 

5 

3 

12 

V  e  r  b  e  x 

A 

1 

75 

6 

1 

B 

3 

42 

6 

2 

C 

4 

1  CO 

6 

D 

2 
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4 
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A 

1 

41 

13 

1 

3 

3 

83 

7 

2 

C 

a 

59 

13 

3 

D 

2 

65 

4 

4 

90 


APPENDIX  F 

NWISS/VERBEX  3000  USERS  GUIDE 

INTRODUCTION: 

This  users  guide  v/as  prepared  to  facilitate  cognizant  individuals 
in  training  their  individual  speech  patterns  to  the  Verbex  3000 
voice  terminal.  The  unit  has  a  maximum  vocabulary  capacity  lor 
360  words/utterances,  150  of  which  will  be  trained  in  order  tc 
interface  wargaming  scenarios  resident  on  the  VAX  11/780  computer. 

The  guide  is  divided  into  the  two  (2)  separate  sections  that 
are  representative  of  each  actual  phase  of  training.  The  first 
phase  is  ENROLLMENT,  which  allows  each  word/utterance  to  be 
individually  and  repeatedly  spoken.  The  second  phase  is  TRAINING, 
which  incorporates  the  repeated  use  of  the  phraseology  necessary 
for  wargame  scenario  interaction. 

What  follows  is  a  list  of  specific  procedures  used  to  enable 
the  sytem  for  both  the  ENROLLMENT  and  TRAINING  phases  of  the 
voice  experiment: 

1.  The  machines  (Verbex,  ADM  31,  VT-102,  and  the  speech  unit)  are 
connected  using  the  proper  adapter  as  shown  in  Figure  #1.  (Figure 
#2  shows  the  configuration  to  run  NWISS  on  the  VERBEX  3000)  . 
The  small  machine  to  the  right  of  the  VT-102  has  "VERBEX 
3000"  printed  on  the  top,  and  will  hereafter  be  referrec  tc 
as  the  "Speech  Unit",  so  as  to  not  be  confused  with  the  larce 
VERBEX  3000  unit  located  just  to  the  left  of  the  table.  The 
following  specific  procedures  should  be  utilized  to  effect 
proper  set-up: 

A.  Turn  the  ADM  31,  VERBEX,  and  VT-102  to  the  power-on  position 

(see  Figure  #1)  .NOTE:  Turning  the  ADM  31  on  is  optional 
for  both  phases. 

B.  The  VT-102  and  the  speech  unit  will  automatically  run 
through  a  self-test  (using  the  alphabet) .  Upon  completion 
of  this  procedure,  the  speech  unit  will  read:  "  Booting 
VERBEX  3000  ".  NOTE:  The  VT-102  screen  will  be  a  scrambled/ 
garbled  alphabet  on  occasion;  this  is  still  considered  to 
be  normal  operation  and  will  not  effect  the  follow-on 
VERBEX  enrollment/training  phases. 
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C.  ^ow  complete  the  following  steps  to  enable  the  system: 

1.  Press  the  SET-UP  key  on  the  VT-102. 

2.  Press  the  Set  Up  A/B  (#5)  key  to  set-up  "B". 

3.  Press  the  Transmit  Speeo  (#7)  key press  slcv/ly  snc; 

repeatedly  to  scroll  to  4800. 

4.  Press  the  Receive  Speed  (#8)  key  press  slowly  and 

repeatedly  to  scroll  to  4800. 

5.  Press  the  SET-UP  key  to  exit. 

D.  Kit  the  RETURN  KEY  (of  VT-102)  this  will  move  the 

cursor  to  the  left  side  of  the  screen. 

E.  With  the  cursor  on  the  left  side  of  the  (VT-102)  screen  enter 

the  following  commands  sequentially,  and  wait  for  the 
computer  to  respond  (as  shov/n  in  the  output  column) 
between  entries: 


ENTER  OUTPUT 

dir  <return>  directory  listing 

ed  .asd  to  use  the  edit  .asd  file 

n  to  number  lines 

l,$p  to  print  lines 


F.   If  KWISS  :w:/NWISS  is  on  line  1:   Enter  "  x  ",  and  continue 
with  step  letter  I. 


G.   If  NV7ISS  :w:/NWISS  is  not  on  line  1,  enter: 

1.  2m0  (assuming  it  is  on  line  two) 

2.  l,$p 

3.  x  (  this  saves  the  new  order  ) 

H.   The  prompt  on  the  VT-10  2  should  be   "  -  "    (a  dash) . 
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I.   With  a  "  -  "    as  a  prompt,  enter:   nwiss/nwiss 
J.   The  screen  shoulc  read: 

Monitor:  Version  2.0 

Copyright  (c)  1983,  Exxon  Corp. 

Galcon  3.3  -  Grasp  3.1 
and  the  speech  unit  will  read:  "  Enter  user  ID  =  ". 

2.  From  the  speech  unit  enter  "  00  ",  then  press  the  enter  key. 
This  will  open  the  supervisor  mode.  ( NOTE ;  the  Enter  Key  rust  be 
pressed  after  each  entry  from  the  key  pad  on  the  speech  unit).  If 
you  realize  you  have  selected  an  incorrect  number  for  input  prior 
to  pressing  "  ENTER  ",  you  may  press  ■  CLR  n  (the  bottom  right 
key  on  the  VERBEX  keyboard)  and  then  press  the  correct  number  to 
be  reentered.  The  "  Please  Wait  "  light  will  illuminate,  and 
after  a  minute  or  two  the  speech  unit  v/ill  read:  "Add  New  User?". 

3.  You  will  now  begin  to  scroll  through  Menu  #1  (see  Menu  Listings)  . 
Press  the  NO  button  to  scroll  through  the  Menu  for  familiarization 
purposes. 

4.  To  begin  the  ENROLLMENT  phase  (approx.  20  min.)  Press  YES 
when  the  menu  reads  "Add  New  User?". 

5.  The  speech  unit  will  now  read  "Enter  New  User  ID  =".  Enter 
your  user  number  and  press  ENTER.  NOTE:  Be  sure  not  to  enter  a 
number  already  on  the  enclosed  list  of  used  numbers,  (see  MENU 
GUIDE)  . 

6.  Put  the  Headset  on  v/ith  the  Mike  on  the  left  side  (see  Figure 
#  3).  MIKE  POSITION  IS  CRITICAL  !!  Position  the  Mike  only  a 
little  to  the  left  of  center,  and  slightly  lower  than  the  center 
of  the  mouth.  It  should  be  no  more  than  1/2  inch  (i.e.  about  one 
thumbswidth)  from  the  lips  without  touching  them.  Periodically 
check  the  microphone  for  proper  placement  in  order  to  insure 
consistency. 

7.  When  the  microphone  is  in  place  and  you  are  ready,  start  the 
amplifier  and  the  tape  deck  in  Bay  #2  (in  order  to  interject 
crov/d  noise  at  65  db  into  the  training),  using  instructions 
attached  to  the  tape  deck  itself. 

8.  The  speech  unit  will  prompt  you  to  "SET  the  GAIN". 
It  will  read  "Gain  Now  at  1,  Yes  to  Set  Gain". 

(NOTE:  A  gain  of  1-3  is  OK  —  under  normal  voice  levels,  a 
gain  of  2  -  3  will  be  maintained  ) . 

93 


9.  When  reacy  to  proceed,  press  the  YES  button  and  perform  the 
following  steps: 

A.   Follow  pror.pt  (ie.  say  "One,  two,  three,  four, 
testing" ) . 

E.  The  speech  unit  will  now  reao:  "Gain  is  set  to  (1-4), 
YES  to  set  gain". 

C.  Press  the  YES  button. 

D.  Follow  the  prompts  as  before. 

E.  The  speech  unit  will  now  read:  "Gain  is  set  to  3,  YES  to 
set  gain". 

F.  Press  the  YES  button. 

G.  Eventually  the  speech  unit  will  read:  "YES  to  sample 
backround  noise". 

H.   Press  the  YES  button. 

I.  The  speech  unit  will  now  read:  "Remain  Silent".  At  this 
point  do  not  make  any  additional  noise  other  than  normal 
breathing  (lasts  about  2-3  seconds) . 

J.   The  speech  unit  will  read:  "Yes  to  sample  silence". 

K.   Press  the  YES  button. 

L.  The  speech  unit  will  now  read:  "Remain  Silent".  Again, 
do  not  make  any  aoditional  noise  while  it  is  sampling  the 
silence  of  the  environment. 

M.  The  speech  unit  will  now  read:  "Gain  =  (1-4).  YES  to 
proceed" . 

10.  NQTP; 

A.  If  the  gain  is  1-3,  do  not  press  YES  until  you  are  ready 
to  proceed  with  the  exercise.  Be  aware  that  the  STOP  button 
can  be  used  at  anytime  throughout  the  training  cycle  for 
one  of  three  (3)  possible  reasons: 

1.  If  you  v/culd  just  like  to  take  a  break  for  a  few  minutes, 
press  the  STOP  button  and  you  will  be  prompted  to 
"Continue  Training?".  Siirply  press  YES  when  you  are 
ready  to  continue. 
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If  you  have  mace  a  mistake  on  an  utterance,  press  the 
STOP  button.  Use  the  MO  button  to  scroll  through  the 
menu  (#2)  until  you  read  "Reject  Last  Utterance?". 
Pressing  the  YES  button  will  accept  that  command  arc 
immediately  continue  with  the  next  word  or  phrase. 

If  you  feel  what  you  have  acne  is  completely  unsal- 
vageable  to  this  point,  press  STOP.  Scroll  through 
the  menu  (#2)  until  you  read  "Logout?".  Press  YES, 
and  if  you  are  in  the  ENROLLMENT  phase,  continue  with 
Step  #2.  If  you  are  in  the  TRAINII^G  phase,  continue 
with  Step  #  13  a. 


NOTE; 


Get  comfortable  before  you  start.  Take  breaks  if  you  need  them, 
and  strive  to  be  consistent  with  your  voice  modulation.  See 
Enclosure  #  1  for  a  complete  list  of  the  words  you  will  see 
throughout  the  session.  The  words  will  be  in  order  of  appearance, 
and  the  "less  than  obvious"  annunciations  v/ill  be  annotated  for 
standardization  purposes. 

If  the  gain  =4,  it  is  best  to  start  again.  One  way  to  do  this  is 
to  continue  as  normal  until  the  actual  cycle  has  started  prompting 
you  with  the  first  word  of  the  session.  At  this  point  press  the 
STOP  button  and  scroll  through  the  menu  until  you  read  "Logout?". 
Press  YES.  The  speech  unit  will  read:  "Exit  (your  ID  #)" ,  and 
will  then  read:  "Logout  supervisor",  followed  by:  "Enter  User 
ID".  Continue  from  Step  #2  using  your  same  initial  ID  number  when 
responding  to  the  "Add  New  User?"  prompt. 

11.  Upon  completion  of  the  enrollment  phase  you  will  see  "Enrollment 
Done.  YES  to  Save".  Push  the  YES  button  and  wait  for  the  "Patterns 
Saved.  YES  to  Train"  prompt  to  appear.  Nov/  push  the  following 
buttons  in  the  sequence  below: 

A.  Push  the  STOP  button. 

B.  "Resume  Enrollment?";  press  the  NO  button. 

C.  "Reject  Last  Utterance?";  press  the  NO  button. 

D.  "Save  Patterns?";  press  the  NO  button. 

E.  "Logout?";  press  the  YES  button. 

12.  This  concludes  the  ENROLLMENT  phase  of  the  individual  voice 
training  cycle. 
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13.  The  next  phase  is  TRAINING  and  it  includes  ten  (10)  different 
sections.  You  will  be  asked  to  train  different  combinations 
of  the  v/ords  you  used  in  the  ENROLLMENT  phase.  The  names  and 
approximate  "time  to  complete"  are  listed  below: 


A.  KWISSGRAM1 

B.  FORCE 

C.  DISPLAY 

D.  POSITION 

E.  TRACK 


38  min. 
27  min 
13  min 

6  min. 

7  min. 


6.  FIRE 

7.  SHIPORDER 

8.  AIPORDER 

9.  DIGITS 

10.  LAUNCH 


17  min. 

2  4  min. 
31  min. 
9  min. 

3  2  min. 


NOTE:  You  may  do  these  in  any  order. 


The  following  procedures  should  be  adhered  to  for  each  section  of 
the  TRAINING  phase: 

A.  When  the  speech  unit  reads:  "Enter  user  ID",  you  will 
enter:  "00".  This  procedure  logs  on  the  "supervisor" . 

B.  The  speech  unit  will  then  read  "Add  New  User?"  (as  part  of 
menu  #1) . 

C.  Scroll  through  the  menu  with  the  NO  button.  Stop  scrollina 
at:   "Retrain  Old  User?". 

D.  Press  YES.  You  v/ill  now  be  prompted  for  your  ID 
number.  Enter  the  same  number  as  before.  The  speech  unit 
v/ill  now  read:  "Loading  Patterns  for  user  IE  (your  ID  #)". 

E.  After  a  minute  or  so,  the  speech  unit  v/ill  read:  "All 
Grammars?".   Always  respond  by  pressing  the  NO  button. 

NOTE:  If  you  press  YES,  you  v/ill  be  asking  for  a  long  exercise. 

F.  The  speech  unit  will  read  a  menu  (#4)  of  all  ten  (10) 
selections.  Scroll  through  it  with  the  NO  button  until 
you  find  a  section  you  v/ish  to  train. 

G.  When  you  find  it,  press  the  YES  button. 

H.  The  speech  unit  will  prompt  you  to  Set  the  Gain.  Do 
this  exactly  as  before  (Step  #  &).  Remember,  be  ready  to 
start  speaking  immediately  after  pressing  the  YES  button 
in  response  to  the  "YES  to  proceed"  prompt.  Examples  of 
word  combinations  are  listed  in  Enclosures  (2). 
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I.   As  in  the  ENROLLMENT  phase,  you  will  be  able  to  use 
the  STOP  button  for  the  same  reasons  as  stated  in 
Step  #10. 

J.  At  the  completion  of  each  of  the  ten  (10)  training  sections, 
the  speech  unit  will  read:  "YES  to  save". 

K.   Fress  the  YES  button. 

L.   The  speech  unit  will  now  read:   "Patterns  Saved.  YES 
to  Test". 

K.   Press  the  YES  button. 

*»  NOTE  **  Test  Sessions: 

1.  For  each  completed  and  saved  training  section  (except 
"DISPLAY"),  you  will  be  prompted  to  say  one  word  or 
phrase;  usually  it  will  be  the  word  " Control_K" . 
Respond  using  the  same  annunciation  and  voice  modulation 
as  you  did  during  training.  The  speech  unit  will 
score  your  training  ability  on  a  scale  from  0  to  100. 
Acceptable  scores  range  from  80  to  100.  Lower  scores 
are  an  indication  that  possible  retraining  is  v/arranteo. 

2.  After  the  DISPLAY  section  is  completed  and  saved,  a 
similar  but  longer  test  will  be  given.  Each  response 
v/ill  be  scored,  as  well  as  an  average  score  given  for 
that  particular  section.  The  test  will  take  abcut  1 
minute. 

14.  At  the  completion  of  each  test  session,  the  speech  unit  will 
read:   "Score  =  .  YES  for  Menu". 

15.  Press  the  YES  button.  The  speech  unit  will  now  reac:  "Exits 
",  then:   "Add  New  User?"  (Menu  #1). 

16.  At  this  point  if  you  would  like  to  continue  with  another 
section  of  the  TRAINING  Phase,  simply  scroll  through  the  Menu 
(using  the  NO  button)  until  the  speech  unit  reads:  "Retrain  Old 
User?".   Nov;  continue  with  step  13d. 

17.  If  instead  you  woula  like  to  log-out  at  this  point,  scroll 
through  the  Menu  until  the  speech  unit  reads:  "Logout?".  Fress 
the  YES  button.  The  speech  unit  will  now  read  (in  respective 
order):  "Exit  ID  (your  ID  #)  ;  "Logout  supervisor"  ("00");  and 
"Enter  User  ID". 

18.  If  you  decide  to  continue  and  interface  with  the  NWISS 
wargaming  scenarios,  the  system  set-up  will  have  to  be  modified 
in  accordance  with  Figure  #2,  and  then  you  can  continue  with  step 
#20. 
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19.  To  shut  the  system  down,  simply  place  all  of  the  individual 
power  switches  (On/Off  switches)  to  the  OFF  position. 

20.  With  the  VT-102  ana  the  ADM  31  establish  the  initial  set-up 
as  per  procedure  1  (A-J) .  The  speech  unit  should  new  read 
"Enter  User  ID:".   Enter  ycur  ID  number  and  press  ENTER. 

21.  The  speech  unit  will  now  reac  "Loading  Patterns  for  IE  =  ". 

22.  You  can  set  the  gain  (as  per  #8  -  1C  )  either  prior  to  or 
after  the  NWISS  is  brought  online. 

23.  Upon  completion  of  the  came,  press  the  STOP  button.  Scroll 
through  the  Menu  #3,  and  follow  steps  17  and  19. 
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HELPFUL  HINTS 

1.  Use  the  enclosed  V/crclist  as  a  guide  while  going  through  the 
EKROLLIIEK'T  Phase.  It  would  also  be  very  beneficial  to  have  studied 
it  prior  to  attempting  the  pftase,  as  the  words  are  presentee  toe 
fast  to  think  about  pronunciation  prior  to  speaking  them. 

2.  Sit  comfortably;  preferably  directly  in  front  of  the  speech  unit. 

3.  Be  ready  to  push  the  STOP  button  if  you  make  a  mistake. 

4.  Study  the  wore  list  (Enclosure  1)  prior  to  enrolling.  Especially 
consider  the  words  in  the  "pronounce"  column.  Say  then  naturally. 
DO  MOT  pause  for  any  word  in  the  ENROLLMENT  phase.  The  spaces  in 
that  column  are  to  help  determine  the  proper  annunciation  of  the 
word.  It  will  be  necessary  to  pause  in  the  TRAINING  phase,  but 
only  long  enough  to  distinguish  between  words. 

For  example,  ("FOR  KITTY"). 

5.  In  general,  use  the  most  natural  pronunciation  that  comes  to 
mind. 

6.  Pronounce  numbers  which  appear  as  part  of  fixed  identifiers, 
in  a  natural  manner  (i.e.  F14A  as  F  Fourteen  A;  MK48  as  Mark 
Forty-Eight,  etc.). 

7.  Fully  pronounce  cigits  which  appear  as  part  of  variable  strings 
(i.e.  Altitude  2500  as  Altitude  Two  Five  Zero  Zero,  etc.). 

8.  Be  consistent  with  your  pronunciations. 

9.  The  Select  YES/NO  light: 

A.  This  light  is  on  the  console  of  the  speech  unit.  Ivhen  it 
is  illuminated  it  means  press  the  YES  or  NO  button  as 
applicable. 

B.  NOTE:  This  procedure  is  only  true  if  the  speech  unit 
reads  a  line  of  a  "  Menu  ".  At  all  other  times,  only  the 
YES  button  will  be  accepted. 
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MENU  GUipE 

NOTE;   Do  net  use  the  fcllcv/ing  list  of  ID  numbers:   00  through 
10  (inclusive) ,47,69  70  or  74 

MENU  #1  MENU  #2 

This  menu  is  invoked  by  entering      This  menu  is  invoked  by 

the  supervisor  mode  (i.e.  00  into     by  use  of  the  STOP 

the  speech  unit)  button  v/ithin  ENROLLMENT 

Phase. 

Add  New  User?  Resume  Enrollment? 

Test  Old  User?  Reject  Last  Utterance? 

Re-Enroll  Olc  User?  Save  Patterns? 

Re-Train  Old  User?  Logout? 

List  User? 

Delete  Olc  User? 

Logout? 
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MENU  S3 

Invoked  by  coir.pleting  the  test 
session  in  the  TRAINING  phase 


MENU  #4 

Invoked  by  comr.encer.ent 
of  the  train ina 


Continue? 
Calibrate? 
Restart? 
Locout? 


All  Granr.ars? 

NWISS1? 

Force? 

Display? 

Position? 

Track? 

Fire? 

Shiporder? 

Airorder? 

Digits? 

Launch? 
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WORD 

Execute 

PRONOUNCE 

1. 

2. 

BAIRASW 

B  AIR  ASW 

3. 

BCAP2 

BEECAF2 

4. 

EC  API 

EEECAP1 

5. 

F14STKCAP 

F  FOURTEEN 

STRIKECAP 


6. 

A6STRIKE 

,  A  SIXSTRIK 

7. 

Control_k 

8. 

Force 

9. 

Track 

10. 

Position 

11. 

Designate 

12. 

Neutral 

13. 

Friendly 

14. 

Enemy 

15. 

Display 

16. 

Bearing 

17. 

For 

18. 

1POINT1 

1.1 

19. 

Sprua 

SPRUANCE 

20. 

Misawa 

KISAWA 

21. 

Losan 

LOS  ANGELES 

22. 

Rathb 

RATH BURN 

23. 

Omaha 

OMAHA 

24. 

Wichi 

WICHITA 

WORDS  TO  BE  TRAINED  ON  THE  VERBEX 

WORD  PRONOUNCE 

25.  Wilso  WILSON 

26.  Knox  NOX 

27.  McCor  KcCCRMICK 

28.  Kitty  KITTY 

29.  VTO  VEE  TEE  ZERO 

30.  0  ZERO 

31.  1 

32.  2 

33.  3 

34.  4 

35.  5 

36.  6 

37.  7 

38.  8 

39.  9  NIKE    (not  niner 

40.  VSO  VEE  S  ZERO 

41.  VHO  VEE  H  ZERO 

42.  VE1  VEE  E  1 

43.  VEO  VEE  E  ZERO 

44 .  VWO  VEE  W  ZERO 

45.  VKO  VEE  K  ZERO 

46.  VFO  VEE  F  ZERO 

47.  VAO  VEE  A  ZERO 

48.  SHI  SHI 

49.  MP6  M  PEE  SIX 
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50. 

Viae 

77. 

BA0 

BEL  A  ZERO 

51. 

Petro 

PETRO 

78. 

Tor.ihk 

TOMOHAWK 

52. 

Alexs 

79. 

Hrpon 

HARPOOK 

53  . 

Wonsa 

WAKDSA 

80. 

Cruise 

54. 

Erase 

81. 

ASROC 

AS  ROCK 

55. 

Plot 

82. 

MK46A 

MARK  FORTYSIX  A 

56. 

LOB 

LOB 

83. 

MK46 

MARK  FORTYSIX 

57. 

ESM 

E  S  M 

84. 

MK48 

MARK  FORTYEIGHT 

58. 

Sonar 

85. 

Torpedo 

06. 

Drop 

86. 

At 

AT   (cne  word) 

60. 

Cancel 

87. 

Range 

61. 

Grid 

88. 

Orders 

62. 

Circle 

89. 

Take 

63. 

XMARK 

X  mark 

90. 

DECM 

DEC  M 

64. 

All 

91. 

Off 

65. 

Place 

92. 

On 

66. 

Center 

93. 

RBOCK 

ARBOCK 

67. 

Radius 

94. 

Blip 

68. 

- 

TACK 

95. 

Launch 

69. 

N 

NORTH 

96. 

Weapons 

70. 

E 

EAST 

97. 

Free 

71. 

W 

WEST 

98. 

Submarine 

72. 

s 

SOUTH 

99. 

Surface 

73. 

BUO 

BEE  YOU  ZERO 

100. 

Air 

74. 

BSO 

BEE  S 

ZERO 

101. 

Tight 

75. 

BPO 

BEE  P 

ZERO 

1C2. 

EMCON 

MCON 

76. 

BEO 

BEE  E 

ZERO 

10  3. 

Surf 

SURF 

104. 

AIRS 

AIPS 

130. 

SSC62 

c 

S  0  SIXTYTLG 

105. 

AEW 

A  E  W 

131. 

SSC53 

S 

S  C  FIFTYTKREE 

10  6. 

Silen 

SILENT 

132. 

SS047 

s 

S  O  FORTY SEVEN 

107. 

Raciia 

RADIATE 

13  3. 

A6E 

A 

SIX  E 

10  8. 

Fire 

134. 

A7E 

A 

SEVEN  E 

109. 

Station 

135. 

E2C 

E 

TUG  C 

110. 

Per  iscop< 

^ 

136. 

EA3B 

E 

A  THREE  B 

111. 

Depth 

137. 

EA6B 

E 

A  SIX  B 

112. 

Course 

138. 

F14A 

F 

FOURTEEN  A 

113. 

Speed 

139. 

F14T 

F 

FOURTEEN  T 

114. 

Proceed 

140. 

KA6D 

K 

A  SIX  D 

115. 

Barrier 

141. 

P3C 

PEE  THREE  CEE 

116. 

Mission 

142. 

S3  A 

S 

THREE  A 

117. 

ASW 

ASW 

143. 

SH3H 

S 

K  THREE  H 

118. 

SURCAP 

SIRCAP 

144. 

SH2F 

S 

H  TWO  F 

119. 

Search 

145. 

HARP 

HARP 

120. 

CAP 

146. 

MK8  2 

MARK  EICHTYTWG 

121. 

Airtanker 

147. 

MK8  3 

MARK  EIGHTYTRREE 

122. 

STPCAF 

STRIKECAP 

148. 

M.K84 

MARK  EIGHTYFOUR 

123. 

Strike 

149. 

Phenx 

PHEENIX 

124. 

Cover 

150. 

Shrik 

SHRIKE 

125. 

Altitude 

151. 

Spar 

SI 

? ARROW 

126 . 

Bingo 

152. 

Swdr 

c 

EDEWINDEK 

127. 

Refuel 

153. 

Walli 

WALLEYE 

128. 

VK00 

VEE  K  ZEROZERO 

154. 

Load 

129. 

SQ53  8 

S  Q  FIVE 
THIRTYEIGHT 

155. 

Stop 

ENCLOSURE 

(1) 

104 


TRAINING  PHASE:  EXAMPLES  OF  WORD  COMBINATIONS 


NWISSQRAMJ 

FORCE 

DISPLAY 

Bearinc  Track 

VE-10  0 

Radius  3  4 

ES-008 

VF-101 

19  5 

VT-C01 

VK-003 

Erase  LOE  ESM 

BAIRASW 

VF-C25 

Plot  LOE  Sonar 

Designate  Neutral 

VH-C04 

Cancel  Circle  All 

Display 

Spruance 

Place  Grid  Track 

Execute  F14STECAP 

Aleks 

Center  Force 

VF-068 

Vlad 

Cancel  Circle  Posit 

Position 

Misawa 

Place  Grid  Force 

SH-105 

Wilson 

Place  XMark  Force 

For  lFointl 

MP-600 

Place  XMark  Posit 

For  Knox 

Rathburn 

Center  Position 

POSITION 

TRACK 

FIRE 

BU-059 

8  4  9 

5-5 

BU-009 

2  Tcr.ihk  Cruise 

2  8  W 

BE-026 

Bearinc  5  7 

1  8  E 

BA-034 

3  ASROC  Torpedo 

13  4 

BP-047 

At  Aleks 

4  0  S 

BS-012 

At  Wonsa 

7  9- 

BE-039 

15  0 

6-2 

BA-057 

2  Torch k  Cruise 

-  35 

BP-C65 

4  Hrpn  Cruise 

9  7- 

BS-017 

5  MK46A  Torpedo 

ENCLOSURE 
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SHIPORDER  AIRORDER  DIGITS  LAUNCH 

Free  Enemy  Surf  Free  Energy  Air  SQ-538  KA6D  F14A  7 

Course  4  5  Weapons  Tight  0  0  9  0  6  MK46A 

DEC  M  Off  Bingo  651  Froceec  Fes  it 

Speed  3  2  Refuel  VK-000  SSC-53  E2C  S3A  8 

Weapons  Tight  6  7  8  5  6  4  A7E  A6E  5 

1  C  Mission  Strike  Control_K    3  2  Sictewndr 

Free  Enemy  Sub  Speed  4  9  3  7  5  0  4  SSQ-62 

9  17  Mission  Cap  SSQ-47  Stop 

ARBOC  Mission  AEW  16  0  2  1  MK  84 

Free  Enemy  All  Cover  2  4  6  F14A  SF2F  7 

D  E  C  M  On  Mission  Search  8  3  6  KA6D  F14T  9 

Course  5  4  Mission  Strkcap  9  4  5  4  3  S0538 


i  or. 


Configuration  to  Train 
Voice  Patterns  on 
the  Verbex  5000 


VT-102 


ADM  31 


SPEECH  UNIT 


""DEO 


** 


THE  ADAPTER  IS  STAMPED:  "  VERBEX  ONLY 


NOTE:  THE  DIAGRAM  DEPICTS  VIEWING  THE 
MACHINES  FROM  THE  REAR. 


FIGURE  *  1 
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SPEECH  UNIT 


Configuration  to  Run 
NW155  on  the 
Verbex  5QQQ 


VT-102 


ADM  31 


"'□CD 


NULL 
MODEM 


*# 


THE  ADAPTER  IS  STAMPED:  "  VERBEX  ONLY 


NOTE:  THE  DIAGRAM  DEPICTS  VIEWING  THE 
MACHINES  FROM  THE  REAR. 


FIGURE  *2 
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NOTE; 


MICROPHONE  POSITION  IS  CRITICAL,  THE 
MICROPHONE  SHOULD  BE  NO  MORE  THAN   1/2 
INCH  FROM  THE  LIPS  WITHOUT  TOUCHING  THEM. 
(I.E.  AS  SHOWN  IN  THE  FIGURE  ABOVE) 


THE  MICROPHONE  POSITION  SHOULD  BE  OEHLtf 

SSLUSEHELS?  lower  than  the  center  of 

THE  MOUTH . 


FIGURE  *3 
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APPENDIX  G 


MACINTOSH  MICROCOMPUTER  USERS  GUIDE 

The  purpose  in  accomplishing  this  training  is  to  prepare  to 
play  various  KWISS  War  game  Scenarios  as  an  Aircraft  Carrier 
Battle  Group  Commander  curing  the  course  of  the  individual  record 
trial.  Commands  for  the  game  will  be  entered  using  the  point 
select  method  to  build  acceptable  commands  in  the  place  of  a 
conventional  keyboard. 

To  accomplish  the  necessary  training,  utilize  the  follov/ing 
steps  to  attempt  to  gain  some  skill  in  building  commands: 

1.  Reach  around  to  the  left  side  of  the  MAC,  and  turn  it  on 
using  the  rocker  switch  on  the  upper  left  side  of  the  back.  Place 
the  mini-disk,  label  up,  in  the  slot  on  the  lower  right  of  the 
front,  with  the  metal  tab  entering  the  slot  first.  (NOTE:  It  is 
not  necessary  to  open  the  tab  first  as  it  is  done  automatically 
by  the  drive.)  Wait  for  the  system  to  load  itself  (which  may  take 
a  little  over  a  minute) . 

2.  You  will  see  a  desk  top  with  tv/o  or  more  file  icons.  Usina 
the  mouse,  move  the  arrow  to  the  icon  labeled  "Offline  Training", 
and  with  the  arrow  placed  anywhere  on  the  icon,  click  the  mouse 
button  twice.  The  program,  which  conditions  the  MAC  as  an  NWISS 
terminal,  will  now  load  and  run  itself. 

3.  When  the  load  is  complete  you  will  see  a  series  of  column 
headers  along  the  top  of  the  desk  top  (i.e.  "NR"  contains  numbers, 
etc.).  At  this  point  you  are  actually  corresponding  with  a 
program  written  in  BASIC.  This  program  provides  menus  of  command 
words  from  a  limited  vocabulary.  It  supports  sequential  selection 
of  these  words  and  their  associated  display  (in  an  orderly  fashion 
on  the  screen)  in  order  to  allow  the  build-up  of  full  commands. 
The  program  also  accomplishes  communications  with  the  VAX  and 
returns  information  from  the  VAX  to  the  screen;  sinilar  to  the 
output  provided  by  the  VT100  terminal.  To  select  a  word  from  a 
menu  use  the  mouse  to  position  the  arrow  on  the  menu  title  along 
the  top  of  the  screen,  push  dev/n  the  mouse  button,  and  hold  it 
down.  The  menu  will  extend  downward  in  its  own  window  (see  the 
attached  example  menus) .  Continue  to  held  the  mouse  button  down 
while  you  move  the  mouse  to  position  the  arrow  on  the  woro  of 
your  choice.  When  the  arrow  is  over  each  word  it  will  be  highlighted. 
To  select  one  of  the  words  simply  let  up  on  the  button.  The 
window  will  disappear  and  the  word  will  be  displayed  on  the  desk 
top.  When  a  command  is  complete,  move  to  the  right-most  column 
and  select  "EXECUTE  ORDER"  from  this  menu.  The  VAX  will  respond 
to  the  order  when  the  system  is  "on-line"  in  the  future,  but 
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realistic  practice  sessions  can  still  be  obtained  in  the  "off-line" 
r;,oce. 

4.  Notice  that  the  right-most  menu  contains  a  selection  for 
"foANUAL  EKTRY".  This  allows  you  to  type  in  the  words  that  arc- 
otherwise  not  on  the  menus.  The  key  under  your  left  palm  with  an 
infinity  sicn  is  the  control  key.  Use  the  delete  key  to  back  out 
errors  in  this  mode.  Cnce  you  have  selected  this  mode  ,  you  must 
then  type  your  desired  word  and  close  the  session  with  an  "*" 
which  will  take  you  back  to  mouse  control.  Kcw  "EXECUTE  ORDER" 
must  be  selected  from  the  last  menu  in  order  to  complete  the  command. 

5.  To  terminate  the  session,  select  "QUIT  PROGRAM"  on  the 
right  most  menu.  At  the  next  window  under  the  FILE  menu  select 
"QUIT",  and  under  the  next  window  under  the  FILE  menu  select 
"EJECT".  When  the  mini-floppy  disk  is  ejected,  turn  the  machine 
off  with  the  switch  in  the  back. 
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BASIC  SCREEN 


NUMBER  MENU 


ENCLOSURE  1 
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|     «     NR 

JJJflODR    ELEK    MUMT    flCFTI     RCrT2    HJPS.TKS    OSG/TGT    CTl 

..... 

PLOT  LOB  ... 
ERRSE  LOB  ... 

iin   iiomei  urriiNL  <               

...  ESM 

CENTER  Ion)... 
FORCE  ... 
POSITION 
TRACK  ... 
DROP  ... 
OLO  ... 

Place  GRID  ™ 
Place  CIRCLE  ... 
Place  HMflRK  ... 
Cancel  GRIO  ... 
Cancel  CIRCLE  ... 
Cancel  HMHRK  ... 
SHIFT  ~ 

HELP  !!! 

3 

GRAPHICS  MENU 


*    KR    GPH 


ELEH    MUMT    RCFTI    RCFT2    LUPS/TKS    OSG/TTJT    CTl 


fOH  ... 

...  FORCE  _ 

Constellation  ... 

Poul„. 

Adams  _ 

Cincinattl  M 

BarDle  M 

Roanoke  «. 

toldsborough  ... 
...  MP7  ... 
...  SH7 ... 
...  UR7  ... 
...  UF7  ... 
...  UK7... 
...   ULU7  ... 
...  TG  I  POINT  2... 


HELP!!! 


amer  OFFLINE 
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t     NR     GPH     RDnRgTTTJlMDMT     RCTTI     RCFT?     mPS/TICS     DSG/TGT    CTL 


.  tMLON    ... 
.   RRDIRTE 
.   SltENT 

.   TRUCK  TVPIS   .. 

.   RAO... 

.   RtO... 

.   RPO... 

.   HSO... 

.   RUO... 

..    OECM   ... 
.   RBOC   ... 
..    BLIP   ... 

ON 

OFF 


fifFUSE  = 


ELECTRONIC  MENU 


t     NR    GPH    ROOR    EUK|        [j|  RCFTI     flCFT2    tUPS/TKS    OSG/TGT    CTl 

?■-;.. |           _=g;  ^ 

...  COURSE  ... 

s^r       ■■-     |          <_^==z 

...  SPEED  ... 

...  STRTION_ 

Foi  Suhimuinrs 

...  SURFRCE 

...PERISCOPE 

....  DEPTH  ... 

MlSCHItlllPOUS 

...  RiinuuE  ... 

...  COUER  ~ 

...  HRNOOUER  ... 

...  PROCEED  COURSE... 

...  PROCEED  POSITION  ... 

...  RtPQRT  SURFACE   ... 

...  RFPnFiTRin  ... 

...   RlPdfll  SIJHMRRISE   ... 

...   OROHIS 

HFIP    !!! 

B 

MOVEMENT  MENU 
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C     NR     CPH     ROOR     ELEH     MUMT 


RCFT2     LUFS/TKS     DSG/TCT    CTl 


SDS 


NliUSS   Ti 


LHUNLH 


LORD 


...   R6E  _ 
...   E2C   .- 
...   rHfl  ... 
...  KR60  ... 
...  P3C  ... 
....  SH2F  ... 

..  MK85   ... 
PHOtSIH   ... 
SHRIKE  ._ 

SPRRROLU... 
SIDEWINOER 
UiRLlEYE   ... 
IIRRPOON   ... 
MK  Ab  ... 


REFUEL  ^. 

BINGO  TO 
.  STOP 
HEIP   !!! 


AIRCRAFT 1  MENU 


t     NR    CPH    RDOR    ELEK    MUMT    RCFTI  1         3k!lirS/TKS    DSG/TCT    CTl 

...  CQVF*\->  . 
...  MISSION  _ 

r  • 1 

p— rr. 

...  CAP 

...  RIRTRNKER 

...  SEARCH 

...   STRIKE  CRP 

...   SURUIELLRNCE 

...   STRIKE 

TAKE- 

COUER ...      • 

AITITUOE  - 

STOP 

HEIP    !!f 

AIRCRAFT2  MENU 
ENCLOSURE  4 
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C     NR     CPH     RDOR     EUH     MUMT    RCFTI     RCrT2 1  ^fiT7Tl0SC/TGT     CR 


NLLUSS  lromer  Of  l 


UJtnPOrJiTlGHT 
WEAPONS  FREE   . 
...   AIL   ... 


FIRE 


TRKE 


.  AIR 
SURFACE 
SUBMARINE 
ENEMY 

...  HRPON  CRUISE 
.  MK48  TORPEDO 
.  ASAOC  TORPEDO   . 


...  BRO_ 
...  BEO_ 
...  BPO_ 
...  BSO_ 
...   BUO_ 

BERRING  ... 

RANGE   ... 
HE1P    !!! 


WEAPONS/TRACK  MENU 


•     NR    GPH    RDOR    ClfK    MUMT    RCfTl     RCFT2    WPS/TKS 


|  •'  'i    ':? 


ctl 


NUJISS  Trainer  OFFLINE 


OESIIjInEmV 

OESIGFfllENDLV  ... 

0ESI6  NEUTRAL... 

DESIG  UNKNOWN  ... 

HT  ... 

...  BIEKSKEYEU 

...  PETHOPRUIOUSK 

...  UJONSON 

...  ULROIUOSTOK 


HMP    "' 


DESIGNATING  TARGET  MENU 
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*     N'R    GPH    ROOR    EirH    Mt»M7    RCTTI     RCrT2    U.IPS/TKS     OSG/TGT  Ej 


NIDISS  Trainer  OFFLINE  | 


EKECUTE  Order 
ConK  Order 
Manual  Input 
HELP!!! 

DANKER. - 

do  nol  u$e  Qurt.  during 
program  run  ! 
OUIT  PROGRAM 


CONTROL  MENU 


ENCLOSURE  6 
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